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I NTRODUCT ION 



Mori than anything ©Ise, a college or university needs visible ways of 

saying forcefully to its faculty, *'Yes, we want good teacherSp we sincerely 

respect good teaching, and we have specific ways to recognize and reward 

it J' This paraphrase of Kenneth Eble (1972) comes closest to saying in a 

few words what this study was about, Gorovitz (1972) has also stated the issue 

pa rt i cu 1 a r 1 y we II * 

This Is the crux of many a matter^^the problem of quality 
controK It Is often argued that the only security for high 
quality in the university Is high quality in its faculty. 
This point, commonly cited in defense of the "star system". Is 
well taken. It falls short of the mark, however* in that the 
quality of professional ability alone does not ensure activity 
of high quality In pursuit of institutional objectives. Thus 
the system of incentives within the university again becomes 
cruc I a 1 , 

How can an administration, having articulated a vision 
worth pursuing, and blessed with a faculty of high quality, 
catalyze constructive change by modifying the system of 
i ncen t i ves^^ the reward structure? The question almost 
becomes a dilemma when one recalls that the reward structure 
although often lamented by the faculty, Is tenaciously per- 
petuated by them as they play out over and over again the 
set piece of passing judgment on their colleagues (pp. SSS-^ 
589). 

If one were unwilling to take on faith that college teaching is of 
high quality, on what basis could one determine it? Such a question was 
posed early In 1972 within the Oregon State System of HIgheT Education by the 
authors. The research procedure was relatively s tra i ght forwa rd i find out 
what policies and procedures are currently used to ascertain the quality of 
Instruction. If satisfied that quality could be ascertained, then one could 
proceed to evaluate the levels and make value judgments (good, uncertain, 
bad). Those value Judgments of "uncertain or bad" could in turn lead to 
corrective action, and thus ensure an improved learning environTOnt for 
students . 
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A research plan was devised v/hich called for interviews with high level 
administrators (provosts, deans^ department heads) on six OSSHE campuses', 
and an analysis of related campus documents* Fifty-two administrators (over 
901 of those selected) were formally interviewed, and policy and guideline 
documents were collected from each campus. Three tentative conclusions were 
reached, all of which supported the findings from the comprehensive 
studies of Eble (ISJO, 1972) and Miller (1972). First, there was no evidence 
of campus-wide systematic evaluations of faculty Instruction found anywhere. 
The evidence used on each campus was derived primarily from secondary sources 
using unsystematic procedures, Secondj there was no way to determine how much 
influence these informal evaluations of teaching had on promotion, salary, or 
tenure decisions. Thus, claims that teaching carried "more weight" or was 
"the most Important factor" in such decisions could not be ascertained from 
available data. Third, there were great inconsistencies within and between 
the campuses In the clarity of the criteria used to assess faculty teaching, 
which made formal quality Judgments nearly Impossible* 

The findings from this study were discussed on each of the six campuses, 
and these discussions generated three groups of critics, "Group X" tended to 
be Incredulous that anyone v^ould propose the systematic assessment of college 
instruction. They argued that the very spirit and essence of higher education 
would be Injured, If not destroyed, by any such efforts* The professor/student 
and teaching/ learning relationships have an Intangible chemistry that defies 
scientific analysis, they asserted* It was the researchers' Impression that 
this group was in the minority amongst faculty. It was clearly In the minority 
amongst the campus administrators Interviewed, 



The Oregon State System consists of three universities, three colleges, one 
institute of technology, and one health sciences center (dentistry, nursing 
and med i c i ne) , 
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"Group Y'' could be described as those faculty who endorsed the idea of 
formal assessment, but felt the state of the art impedes implementation. 
Their arguments were not addressed as to whether one should evaluate a pro- 
fessor's instructional qualityp but how. Questions about the reliability and 
validity of measures were foremost with themj and their general position was 
that we need much more research on appropriate measures before considering 
them in a systematic decision process. It was the researchers' impression that 
this group was sizeable amongst faculty ^ probably in the majority. They have 
said, "We agree with your basic argument (quality should be systematically 
assessed) j now show us how to do it*" 

Finally, "Group Z" also appeared to occupy a minority status amongst 
faculty. They argued that the objective assessment of instruction was 
absolutely necessaryj and only faculty inertia prevented it. The technologVi 
1 ns t rumen tat i on i and measurement issues have been solved well enoughj and they 
urged Immediate action to implement a more formal faculty evaluation process* 

Our own position was a mixture of positions Y and Z: the technology and 
measurement issues have received enough study to warrant the development of 
tentative models and test situations for eventual Implementation, The state 
of the art was not sufficients howevers to propose "the way" to mea^ re faculty 
performance, In general, or the quality of Instruction, In particular* Rather, 
careful study should be given to the present processes of performance assessment, 
and to the institutional Incentive systems and their implications for present 
and future applications, if any kind of orderly change, such as suggested by 
GrosSi et al* (1971)1 is to take place in higher education. 

The above findings led to a research proposal being submitted to the Fund 
for the Improvement of Pos tsecondary Education (HEW) for financial help to 
carefully study the process of assessing faculty, and a three-^year grant resulted. 
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The present report details the findings from the three years of research 
(September 1973 to September 1976) • 



The Fund for the Improvement of Postsecondary Education is within the 
Office of the Secretary, Health Education and Welfare, Washington, D*C, 
Neither the FUND nor the Secretary necessarily agree with or endorse the 
present findings. 
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METHODOLOGY 

The prasent approach to ^'better'* Instruction did not procead from a 
predetermined solution, e,g., adding madia to the classroom (or some other 
technological form of assistance), or using the PSI teaching method (or any 
other specific instruct lonal method) . Rather, the approach proceeded from 
an analysis of decision making, i,e,, how is good or bad Instruction recog- 
nized, and having done so how Is ft rewarded or punished? Thus the study 
method adopted was one of operations research, where the scientific method 
was used to analyze the decision structure of an ongoing system. 

Because some inconsistencies had been noted in preliminary studies, 
the research staff were unwilling to accept unverified descriptions by campus 
administrators of the promotion process. Astin and Lee (1966) used this 
approach by mailing questionnaires to a national sample of college and univer- 
sity deans, asking them to indicate the frequency of use of various sources 
of information when they evaluated teaching effectiveness, and during the 
course of the present study Seldin (1975) partially repeated the Astln-Lee 
study on a sample of deans in private liberal arts colleges. While many of 
the present findings correspond closely to those in both studies, there are 
also some very important d i screpenci es which would not have been possible to 
detect using the previous methodology* 
Project Objectives 

There were two major objectives: (I) to describe and analyze the factors 
Influencing tenure and promotion decisions; and (2) to develop Incentive models 
capable of positively influencing the quality of instruction. The Results 
section will be devoted mainly to objective one, while the Overview and 
Implications section covers objective two. 

An early comment needs to be made about objective one being phrased broadly 
enough to cover all normal faculty functions. It was clear from our first 



System study that the separation of the instruction function from those 
of research and service was artificial and potentially misleading. Faculty 
evaluations invariably cover ai I "the f unct Ions , though each function might be 
discussed and weighed Individually by a revlav committee or an administrator. 
Hence although the research interests were primarily directed toward the teach 
Ing function* data were collected on whatever evidence was normally introduced 
in tenure or promotion procedures. 

Data sources* A number of data sources were used to address the first 
objective* Administrators (presidents and deans) on each campus were a primar^ 
source* They supplied their policy statements, guidelines, and forms used in 
the assessment of faculty. These data allowed us to describe the context and 
processes relevant for the decision making we wanted to analyze* In addition^ 
the administrators respondedj in individual interviews, to a set of questions 
about their perceptions of the process; e,g., What evidence counts the most 
for each faculty function?; What gaps exist In the evidence provided?; How 
might the process be improved? (see Appendix A for the Interview questions). 
Where campus-wide personnel committees existedj project staff interviewed 
each of the niemberSj using slightly rephrased questions* 

The second major source of data came from several System-wide surveys of 
OSSHE faculty members* These surveys were accomplished by using stratified 
random samples of faculty* (a) by departments (selected by department 
for adequate discipline and institution representation); (b) by campuses 
(selected by individuals for adequate system^wide representation); and (c) by 
professional schools (selected by schools for adequate rank representation). 
In addition, all department chairpersons In the system were invited to respond 
unless they had already done so through random selection* In order for a 
faculty member to have been included In one of these sample groups, he or she 
must have been at least at the rank of assistant professor with a minimum .SO 
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FTE appointment in an Ins truct lona ] area. Details of the sampling methodology 
are i n Appendix B; 

Data collection procedures . A survey form^, called the Faculty Perception 
Questionnaire (FPQ), was developed to obtain Information from faeulty regarding 
their perceptions of the factors likely to be Influential in deciding promotion^ 
tenure cases on each campus (Appendix C has the FPQ and its instructions). 
Thirty-four factors were each to be Judged as to their use or non=use* Those 
marked "used'' were then rated from one (''useds but very minor influence") to 
seven ("very significant influence"). Faculty could also indicate (a) whether 
a factor had Increased or decreased In Influence since 1973^ and (b) the five 
factors each would "prefer" to be most influential. 

Table 1 shows the distribution of 1976 FPQ respondents* from which all 
subsequent analyses have been drawn. These were faculty who had responded in 
either 1974 or 1975 to earlier versions of the form^. The return rate ranged 
from 71% to 95% across the six campuses, with an average rate of 771' 



The FPQ IS the final modification of two previous versions, which were returnee 
by over 1,200 OSSHE faculty In surveys during 197^ and 1975. However, the 
present findings will be based only on FPQ respondents, because we consider 
that version to be the superior of the three* 

The project began In Septembers 1973 and was funded for a three-year period, 
so 1973 was the referent for any observed changes. 

It should also be noted that comparable data collection took place at Oregon 
Institute of Technology and the UO Health Sciences Center, but the data have 
been omitted from this report because of the highly specialized natures of 
the two schools. 
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Table 1 

1976 FPQ Respondents by Academic 
Rank and Type of Institution 





Co ] 1 ege^ 


Un i Vj 


OSSHE 


Assistant Professors 


38 


62 


100 


Associate Professors 


kk 


88 


■ 132 


Professors 


kk ■ 


lit] 


185 


Faculty Administrators 


20 


kQ 


68 


Total 


146 


339 


A85 



Project Impacts 

In addition to the two major project objectives, there were also operational 

changes desired as a result of the project* Five of these were enumeratedj 

shortly after the proJect*s Inceptioni as likely areas for positive impact oni 

I, The amount and kinds of data collected to assess the quality of 
faculty performancej especially as they relate to the teaching 
f unc t ion * 

II. The amount of specification and utilization of procedures for 
assessing faculty performance to make tenure and promotion 
dec i s Ions * 

111. The types -of differential weightings given to faculty functions 
in assessing Individual performance. 

IV. The amount, types and utilization of professional development 

activities, especially as these relate to the teaching function* 

v. The perceptions of students, faculty, and adml n I stratof s regard^ 
Ing the calibre of the educational experience provided. 

Methods of Influence . A plan was developed, in collaboration with outside 

consultants, to try to promote changes wherever the project's findings suggested 

the need for them. The major source of Influence was expected to be from the 

campus reports, which would analyze the present (1973) procedures for gathering 
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evidence on promot ion/ tenure decisions and make recommendations for Improvement. 
Insofar as subsequent project efforts, these would focus on the upper adminis* 
tratlve levels, to assure continued attention to the report and to offer 
technical assistance where needed. It should be impressed upon the reader that 
the intent was primarily to assure genuine attention to, and consideration of, 
the report and not necessarily the adoption of the recommendations, per se. 

The dissemination strategy was the same on each campus. First, each campus 
president and academic dean was sent their campus-specl f ic analysis, in draft 
form, and then asked for a critique session with project staff. After incorpor^ 
atlng any changes due to errors of interpretation, copies of the report were 
then sent to other top level administrators (usually deans). The third step 
consisted of a number of group discussions of the report and its implications 
for faculty evaluation procedures and policy. The fourth step was to distribute 
the report, or in some Cases an abbreviated version, to faculty for their 
consideration. 

This intervention strategy was based on the assumption that the faculty are 
neither sufficiently motivated, nor in a powerful enough position, to affect 
changes in policy and procedures. Rather, the administrators must exhibit 
genuine concern( i ,e, , verbal cofrniitment plus action) about the quality of 
faculty performance before changes can be effected. Thus, if the decision makers 
and the most influential review groups could be made sufficiently uncomfortable 
with existing promotion policy and procedures, then change would be likely to 
occur. Project years one and two were to be largely devoted to the above strategy. 

Data col lection procedures . Three sources of data were used. First, the 
faculty who responded to the first questionnaire survey were asked to respond 
again in 1976, for a b fore=after comparison of perceptions. Second, administra- 
tors on each campus were interviewed regarding specific changes they had perceived 
in the three year period. Third, project staff accumulated anecdotes and 
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serend ipi tous data on any changes which could be reasonably attributed to 
project act i vi t i es. 
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RESULTS 

A general overview of this Results section should be helpful to the reader 
In anticipating what is included. We shall first describe the FP^ findings 
using the trad i t iona 1 categories of faculty performancei viz,, instruction, 
researchi and service* The next analysis will describe the ''most influential" 
types of evidence, regardless of the functional category, and will introduce 
"certainty scores" as a further way to refine the data* Factors found to 
exert the "least Influence" are analyzed next, followed by the data on "preferred' 
factors. Finally, the presently influential and preferred factors are contrasted 
to present a synthesis of the FPQ, data, ' 

Findings from Individual interviews follow the FPQ analyses* The set 
completed at the beginning of the project are presented first, to augment the 
preceedlng FPQ analyses. Interviews conducted In the final year, along with 
some survey data, are used to analyze major Impacts which could be attributed 
to the project, ' 

The findings are always separated for colleges and un i vers i t i es * In addition 
four major academic disciplinary categories are frequently utilized to reflect 
similarities and differences within the two types of campuses. These disciplines 
are Arts/Humanities, Phys ica 1 /Natura 1 Sciences, Social Sciences, and Professional, 

A couple of specific cormients on the data presentations and analyses may 
also be helpful to the reader. The tables and figures have all been placed at 
the end of each subsection, to enhance continuity in the narrative* Nevertheless, 
some switching back and forth between tables and narrative will be necessary. 

The reader may be curious as to why data is presented neither by academic 
rank nor by decisions on promotion to full professor. In the reports given to 
each campus^, findings were displayed by academic rank, giving the average values 

^ CustomlEed reports were developed for each OSSHE campus during the course of 
the project. Each contained analyses based upon data collected at that 
setting, along with a unique set of recontnendat Ions • 
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for each rank by each factor. This procedure has been omitted from the present 
report, because consistent and sizeable differences were rarely shown by rank* 
Associate and full professors tended to be very similar In their perceptions and 
preferences* Assistant professors were frequently In disagreement with them, 
but the differences were not large enough to be important at either campus or 
discipline levels. (The differences were, however, sometimes important at a 
depirtmant level.) Readers interested in data by academic rank may consult 
Appendix 

Insofar as data on factors influencing the associate to full professor 
decision, we found few noteworthy changes in the factors used, or the influence 
attributed to them in the two decisions during our first faculty survey* Some 
might go up one scale point, others down one, but most stayed tha same, suggesting 
that the results of the assistant to associate decision would be sufflclehtly 
representative. This reasoning was also bolstered by our administrator inter- 
views, which consistently pointed to the great Importance of the first decision, 
since tenure Is convTionly linked to It, This decision currently carries about 
a S750|000 salary and fringe benefit commitment for the institutions, if the 
individual teaches at least twenty years. We should, then, have an accurate 
description of what the institution and the faculty perceived as important for 
advancement if we know what It has taken to become an associate professor* 
Rational Grouping of FPQ Factors ^ 

The FPU factors were rationally grouped Into five categories. Three 
encompass the major faculty functions of Instruction, research/scholarly develops* 
rnent, and service. A fourth or "general" grouping includes six factors which 
could not be placed in the above categories, e.g., "support of departmental 
poMcy and goals." The fifth group Is a final exception to the first four, and 
Includes three major sources of judgments: "evaluation by department chairman", 
"evaluation by school/department committee", and "formal and systematic appraisal 
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of the candidate by peers outside of insti tut ion." These ■^composite-' factors 
reflect judgments based upon Idiosyncratic use of particular criteria. 

The rational groupings are shown In Figures 1 through 5. The three 
university and three college campuses differ very little in the importance 
(average FPii scale values) attributed to teaching factors (Figure 1). Evidence 
from student rating scales was perceived as carrying the most influence and 
the assessment of course syllabi the least. The colleges usually gave more 
weight to these than the un I vers i t i es , although this can be somewhat misleading 
because the average scale values are comnxsnly In the '^mlnor influence" range. 

Striking differences appear on the research (scholarly development) factors 
(Figure 2), In all cases the universities attribute more influence to these 
factors, when reaching pronnot Ion/tenure decisions* The publication of material 
in formal outlets is by far the most influential factor (it was highest of all 
34 FPQ factors). 

The factors associated with community and institutional service were not 
perceived as very influential (Figure 3)* Like the teaching factors, the colleges 
are more inclined to give them credence than the universities. Service on 
committees Is the single factor most likely to be used In promotion/tenure 
decisions from this grouping* 

The two special groupings of FPQ factors yielded some of the most influential 
kinds of p romot Ion/tenure evidence. This was especially so with college faculty 
where '^tlme In rank'S "obtaining an advanced degree's and "personality 
traits and attitudes" were viewed by our respondents as considerably more 
influential than in the universities (Figure k) , It was also true that the 
judgments of department chairmen were an important influence, especially In the 
colleges (Figure 5)* The off^campus appraisals appear to be used mainly on 
ynlverslty campuses (though as we shall note later we view this as one of the 
most disingenuous kinds of evidence used), 
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What has the analysis of FPQ factors by traditional faculty functions told 
us thus far? Firstj the research/scholarly development function is Interpreted 
very differently according to the type of campus. The universities seek evidence 
on pubii cat ions to judge performance, whereas the colleges promote the attainment 
of doctoral degrees as appropriate evidence. Second, a large number of factors 
were suggested in the campus documents we collected as being influential, and 
our findings confirm this. Many sorts of evidence are colTected™ but what may 
not have been obvious before Is how little influence most of these really exert 
on promotion/ tenure decisions. Thus precision In understanding the latter may 
be obfuscated by numerous performance indicators which, in fact, exercise minor 
Influence* 
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Figure 1 . A comparison of mean FPQ values on teaching factors of 
college versus university faculty: All ranks and 
disciplines combined. 
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Research Factors 
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Figure 2 , A comparison of mean FPQ values on research factors 
of college versus university facultyi All ranks and 
dj scipl ines combi ned. 
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Figure 3 * A comparison of mean FPQ values on service factors of college 
versus university facultyt All ranks and disciplines combinac 
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Figure ^ . A comparison of mean FPQ values on general factors 

of college versus university faculty: All ranks and 
disclpiines combined. 
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Figure 5 



A comparison of mean FPQ vaiufc= on composite factors 
of college versus unlverslt faculty: All ranks and 
disciplines combined, - 
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Most Influential Factors 

The analysis of FPQ factors by traditional performance functions was not 
entirely satisfactory as a way to understand what kinds of evidence are likely 
to have significant influences on promotion decisions. Another way to analyze 
the findings was to consider extremes of influenccj independent of functions, 
and to group faculty by academic discipline, A rank-orderlng of the 3^ FPQ 
factors from the highest average influence to the lowest clearly established 
that only a small number of the factors were perceived as exerting moderate to 
significant Influence. Table 2 displays the first five rankings for each 
d isclpl Ine, 

The university disciplines all viewed publications as the most influential 
piece of evidence in proinot Ion/tenure decisions, whereas obtaining a doctorate 
was most influential for the four cQllege disciplines. Student ratings were 
perceived as highly influential for all disciplines on both kinds of campuses, 
as was time in rank (one exception). The university Physical/Natural Science 
faculty were unique in perceiving the obtaining of grants as highly signlflcantj 
whereas their college counterparts were unique Insofar as the influence of 
innovative efforts In teaching. Another unique teaching factor was found within 
the university Social Science faculty, who saw the formal appraisal of teaching 
by colleagues as very Influential. Many administrators had pointed out the 
importance of such evidence, but most had also said they were discouraged with 
what was presented In dossiers relevant to such performance. 

These discrepancies, between v^hat Is perceived and what is actually used, 
led us to enquire beyond the average FPQ scale values. For example, we found 
that the variance associated with departmental average scale (FPQ) values was 
sometimes sizeable. It was not rare to find 20-301 of a department indicating a 
factor was not used, another 20-301 saying it was but carried little weight, and 
the remaining ^0-^601 saying the factor carried significant weight 1 



Certa i nty scores . A special procedure was developed to try to make such 
varjabiiity more understandable and potentially useful, and resulted in derived 
values we labeled ''certainty scores." The rationale upon which this certainty 
score concept Is based holds that we are dealing with a set of opinions held 
by faculty members^^oplnlons as to whether or not certain indicators are 
employed in reaching decisions about their performance and. If usedi the extent 
to which the indicator Influences the decision. We are basically interested 
in proportions of respondents answering yes or no regarding each factor-s use. 
As the proportions approach 50-50, the reliability of the response must be 
questioned. I.e., "Does a 50^50 split (1/2 say a factor is used and, thereforei 
rate it, while 1/2 say the factor Is not used) represent: (a) a true reflection 
of the population opinion, or (b) ambiguity regarding the factor's influence?" 

The certainty score analysis focuses on the proportion of respondents from 
a given group indicating the use of a particular Indicator. If In a department 
all faculty members Indicate a factor was used (or not used),' there would be 
little reason to question the Influence (or lack of Influence) of the indicator* 
However, as the number indicating that a factor was used approaches the number 
Indicating that it was not used, the actual Influence of the indicator becomes 
less clear. The derived or "certainty score", was computed by the following 
formu I a : 



CS « 



N 



X 100 



Where K ^ Number of respondents Indicating an indicator was . 
not used, and 

N ^ Total number of persons In the unit 

Consequently, department CS's range from 0 to 50* A low "certainty score" 
indicates that factor's use Is clouded by uncertainty, 

Uncertamty about the use of an Indicator can derive from many sources. 
Lack of awareness of procedures or criteria by Individual faculty members is 
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probably the primary and most obvious source. A second possible source may 
result from department'-speci f ic conditions, i.e., there may be specific 
. departments In which clear communications about Indicators have not occurred. 
A third source could reflect nebulous campus procedures » guidelines and policy 
statements. And finally, uncertainty could reflect some characteristic of the 
item itself. 

The certainty score dimension seemed to add an important perspective to 
the data. The 3^ FPQ factors were arrayed by CS and average FPQ values, to 
see if other meaningful clusters might emerge. Figures 6 and 7 ihow the 
consequence. Three groups of FP^ factors emerged for each type of campus. 
They ware defined as follows: (a) definitely Influential (high mean, high CS)| 
(b) definitely uninf luential (low means high CS); and (c) ambiguous (moderate 
mean , low CS) , 

Returning now to the analysis of the '-most Influential" FPQ factors, we 
can add the dimension of certainty, i.e,, which factors had high mean scores 
and a majority of faculty agreeing on their use? Figures 8 and 9 display these 
factors for colleges and universities. The groupings rely upon different CS 
cutting scores because of the unique ways the factors clustered on the two 
dimensions. These two figures probably give the single best display of which 
factors In this study were perceived as definitely Influential. 

Each type of campus recognized the same two kinds of Influential evidence 
on teaching: student ratings and Informal appraisals. Merged In this way 
neither perceived evidence on innovative teaching nor on formal appraisals of 
teaching, so these may be truly unique to particular disciplines. Two kinds 
of evidence were unique to the collages: obtaining a doctorate and personality 
traits. Three kinds of .evi dence were unique to the universities- evaluation 
by school/department conmittees, public presentation of products, and service 
on Institution/system committees, 
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Table 2 



Five Most Inriuential FPa Factors for College 
and University Faculty by Discipline* 


1 n f 1 uent I a 1 


Rank Wi thi n Col umn 


Arts/ 

Human! t Ies 


Physical 
Sci ence 


Soci a 1 
Science 


Prof. 
School s 




Col, Univ, 


Col. Univ. 


Col. Univ. 


Col. Univ. 


Pub 1 1 CB 1 1 ons 

Obtaining Advance Degree 


1 

I 


1 

1 


I 

3 


1 

1 k 


Grant Support for Research 
Student Ratings 


2 2 


2 

3 5.5 


2 2 


k 2 


Jirne in Academic Rank 




2 k 


1 3 


2 


berviCe on uepa rLrrcnca i womrni ttues 


3 








Public Presentations of Products 


3.5 




k 


3 


Formal Peer Appraisals (from off campus) 


5 


3 






Personality Traits and Attitudes 




5.5 


5 


3 


Informal Colleague Appraisals/Teaching 




k 5.5 


k 


5 


Formal Colleague Appraisals/Teaching 






5 




Service (no fee) to Community 








5 


Innovative Effort in Teaching 




5.5 







Two "composite" factors i evaluations by chairman and by department committee, 
were deleted in order to allow specific types of influential evidence to 
emerge. 
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Figure 6 . Plot of FPQ mean values by certainty score 
for all College faculty: All ranks and 
disciplines combined 
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Figure 7 . Plot of FP(i mean values by certainty score 
for all University faculty* All ranks and 
d i scl p 1 1 nes comb ined 
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Time In Academic Rank 
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Service on Departmental Committees 
Pub Ilea t ions 

Figure 8, All College FPQ values 
and certainty scores ^ 
d I sc i pi 1 nes comb i ned 
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Figure 9, All University FP(i 
and certainty scor( 
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Definitely Un i nf I uent i al Factors 

The smallest clusters of factors displayed In Figures 6 and 7 were 
those to which faculty assigned low FPQ ratings and were in considerable 
agreement that the factors were not used (high CS values) ♦ The descriptors 
for these are In Tables 3 and 

We found that the college faculty saw the least influence being exerted 
by formal methods of appraisal * whatever the professorial function. They also 
agreed with university faculty about the non^use of evidence which could be 
obtained by evaluating course syllabi (e.g., is the content sufficiently 
thorough^ up-^to-date, accurate, consistent with program intent, etc?) and the 
classroom examinations used (e*g., are they fair, and consistent with lecture 
content, syllabus and program object Ives , etc?). 

The thl rd cl us ter , labeled "ambiguous factors'S probably includes factors 
which could be eventually labeled as definitely uninf luential , were efforts 
to be taken to define them more adequately for faculty. College and university 
faculty were found to share thirteen of the twenty-six ambiguous sources of 
evidence (Table 5). Insofar as academic disciplines, there were not great 
differences within or between college and university campuses. (Readers 
interested In such data should see Appendices F and G,) The most uncertainty 
was found in the Arts/Human 1 ties and the least amongst faculty in the Physical/ 
Natural Sciences, regardless of campus type. 
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Table 3 

Definitely Un 1 nf I uen 1 1 a 1 Promotion/Tenure Factors" 
In College Settings for Teaching and 
Administrative Fadulty Combined 



Rating on 7''point Scale 
(N=138) 



Supervision of Theses 


.2 


Formal Colleague Appraisals/Service 


.2 


Formal Colleague Appraisals/Teaching 


.k 


Formal Peer Appraisals (from off campus) 


.3 


Formal Colleague Appraisals/Research 


.2 


Colleague Assessments of Syllabi 


.6 



A For a factor to have been included In this table it must have been rated in the 
bottom ten (ranks 25^3^) by faculty in at least 3 of the following groupings: 
Assistant Professors* Associate Professors, Full Professors, and Chairmen, It 
also must have had a certainty score >^ 30. 
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Table 

Definitely Uni nf 1 uentlal Promotion/Tenure Factors" 
In University Settings for Teaching and 
Administrative Facuity Combined 



Mean Rating on 7"P0int Scale 
FPQ Factor Name /N=9fiQl 



Colleague Assessments of Syllabi 


.k 


Election to Faculty Senate 


.6 


Supervision of Field Placements 


.5 


Formal Colleague Appraisals/Service 


.6 


Credit Hours Production 


.8 


Evidence of Student Learning In Courses 


.8 



For a factor^to have been Included In this table it must have been rated in the 
bottom ten (ranks 25-^3^) by faculty In at least 3 of the following groupings' 
Assistant Professors^ Associate Professors^ Full Professors, and Chairmen. It 
also must have had a certainty score >^ 25* 
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Table 5 

Factors Faculty Are Uncertain About Regarding 
Their Influence m P/T Decislons^^ 



FPQ Factor Settings in which 

Label and Number ^'uncerta I nt y'^ appl tes 



Col 1 eges Un i vers 1 1 1 et 



k Support of Department Policy and Goals / / 

7 Formal Colleague Appraisals/Teaching / 

8 Grant Support for Research ^ / 

9 Supervision of Theses ^ 
)0 Personality Traits and Attitudes / 

11 Consultation Record on/off Campus / / 

12 Service on Institution/System Committees / 

13 Academic Advising ^ ^ 
]k Membership In Professional Organizations / / 

15 Service (no fee) to Community ^ ^ 

16 Supervision of Field Placements / 

17 Elected Offices In Organizations / ^ 

18 Public Presentations of Products* / 

19 Informal Col 1 eague 'Appra i sa 1 s/Research / / 

20 Formal Colleague Appraisals/Research / 

21 Obtaining Advance Degree >f 
23 Informal Col league Appraisals/Service / / 

25 Evidence of Student Learning In Courses / 

26 Election to Faculty Senate / 

27 Effort to Remain Current In Discipline / / 

28 Credit Hour Production ^ 

29 Student Demand for Course / / 

31 Evaluation by School/Department Committee / 

32 Aval labl 1 i ty to Students / >/ 

33 Innovative Effort in Teaching / / 
3^ Formal Peer Appraisals (from off campus) ^ / 



Ambiguous P/T factors have been operationally defined as those FPQ Factor 
with means ranging from 1 to 3-2 and certainty scores < 30 for collages 
and < 25 for universities. 
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Preferred FP4 Factors 

As the reader may recall , all FPQ respondents were asked to nominate up 
to five factors they would prefer to be judged by regardless of the promotion/ 
tenure decision. These data give yet another perspective on the decision 
process* Tables 6 through 13 show faculty preferences along with the FPQ 
factors they saw as influentiaK The preferenceSs by academic disciplines, 
had to be nominated by at least 30% of that group (an arbitrarily chosen 
cutoff point). For example, 511 of the 43 Phys i ca 1 /Natura 1 Science faculty in 
Table 6 Indicated a preference for "effort to remain current in discipline," and 
30% of this group expressed a preference for "Innovative effort in teaching*" 

The college faculty as a whole expressed substantial preference for three 
FPCi factors: effort to rema I n^current In discipline, student ratings, and 
evidence of student learning. At least two of these emerge for each of the 
separate dlsclpllneSi but !n no case is more than one of the three ever 
viewed as "preferred and presently influential*" In other words the agreement 
across college disciplines on the preferred forms of evidence is fairly high, 
but the agreement between preferred and presently Influential forms of evidence 
Is quite low. Moreover, a majority of the factors which are most Influential 
now are based largely on evidence collected In neither systematic nor compre- 
hensive manners (refer back to Figures 8 and 9), 

The university faculty as a whole preferred two FPQ factors i publications 
and student ratlngSp The separate disciplines also viewed both of these as 
preferred and presently Influential* When additional preferences were expressed 
they emphasized forma 1 apprai sa Is by colleagues, whether of one's teaching 
or one's research. Conversely, there appears to be agreement amongst the 
university faculty sampled here that a decrease in the Influence of data based 
on impressionistic and Informal evaluations, and in the subsequent use of these 
data by chairmen and conmittees, would be preferred. 
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In summary^ we found a number of the ''definitely un inf 1 uent fa 1'* FPQ 
factors emerged as preferred factors when faculty were given a choice. These 
preferences emphasize two major sources .of formal and systematic data: (a) evi- 
dence from students regarding their learning and their evaluations of the teach- 
ing; and (b) evidence from colleagues regarding scholarly or artistic achieve-- 
ments. 

Given the fairly clear emergence of most influential, least influential, 
and preferred FPQ factors, we want to return now to the potential usefulness 
of certainty scores In developing more refined meaning from the project's findings. 
As the reader may recall, a high CS value (above 35) means faculty are near 
consensus about the use or non-use of a factor in promotion/tenure ha^'InTs, 
whereas a low CS value (under Ig) means faculty are in nearly total o . j . nent 
over the use or non-use of a factor* 

Preferences and Present Influence . An ideal circumstance for campus de- Ision 
makers would be to have high certainty about preferred factors which are, in 
fact, presently influential. This circumstance was found more often on the 
university campuses than on the college campuses* Publications and student ratings 
of classroom performance are highly Influential, highly preferred, and university 
faculty are near consensus that the factors are used. Conversely, the circumstance 
in which decision makers should be least satisfied is where a highly preferred 
factor has faculty consensus that it is not used. University faculty perceived 
three such sources of either evidence or review (factors 7, 20j and 3k on Table 5) i 
formal colleague appraisals of both research and teaching* 

The college decision makers appear to face much more dissatisfaction in 
trying to apply explicit and uniform criteria for promotion/tenure decisions. 
Like the university faculty there were two FPQ factors (2 and 22) which were 
highly influential, highly preferred, and had high CS values: student ratings 
and evaluation by department chairmen (which Is a composite factor that must 
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rely on idiosyncratic Influences)* However, both college and university 
faculties indicated preferences for three factors that are presently of 
ambiguous use* These were evidence of student learning, currency in discipline 
and innovative effort In teaching amongst the college faculty, and forma] 
appraisals of teaching, informal appraisals of research, and formal peer apprai 
sals (off campus) amongst the university faculty* One particularly disturbing 
feature of this group of factors, insofar as the project's main objectives, is 
that half of them Involve evidence potentially useful for evaluating teaching* 
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Table 6 



A Comparison of Presently Influential and Preferred Factors 
for Physical/Natural Sclencesi College Settings 

(N=43) 



Grouping 


Present 
Rank 


Preferred 
Rank 


Percent 
Indicating 
"Preferred" 


Prefarredi but not presently influential 








u i 1 W 1 w L w n^iiiQ III wwiil^liL lit t^lswlH''^'^ 


> 1 1 ^ 


1 




Preferradj but not influential enoughs 








Innovative Effort in Teaching 


7.5 




301 


Preferred and Influential- 








Evaluation by Department Chairman 


2 


2 




Student Ratings 


^ 


3 


371 


Evaluation by School/Department Committee 


6 


A. 5 


301 


Presently influential ^ but not preferred 
by 301 or more: 








Obtaining Advance Degree 


1 






Time in Academic Rank 


3 






Informal Colleague Appraisals/Teaching 








Personality Traits and Attitudes 


7.5 






Service on Departmental Committees 


9 






Public Presentations of Products 


10 






Exact rank orders were only carried out to 
were actually somewhere between 11 and 3^* 


the tenth 


rank. Any 


marked >_ 1 1 
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Table 7 

A Compirison of Presently Influential and Preferred Factors 
for Social Scfences* College Settings 
' (N^22) _ _ 



Percent 

Grouping Present Preferred Indicating 

Rank Rank "Preferred*' 

Preferred, but not presently influentjah 

m 

Effort to Remain Current In Dticlpllne i ' 1 2 361 

Evidence of Student Learning In Courses >_ 11 3 32% 

Preferred, but not influential enough i 

Preferred and influential 

Student Ratings 2 I 451 

Presently influential, but not preferred 
by 30% or more : 

Time in Acadefnic Rank 1 

Obtaining Advance Degree 3 

Informal Colleague Appraisals/Teaching k 

Personality Traits and Attitudes 5 

Evaluation by Department Chairman 6 

Innovative Effort in Teaching 7 

Service on Departmental Committees 8/5 

Publ Icatlons ' 8,5 

Evaluation by School/Department Committee 10 
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A Comparison of Presently Influentfal and Preferred Factors 
for Art & Humanltlesi College Settings 
(N-19) 


Grouping 


Present 
Rank 


Preferred 
' Rank 


Percent 
Indf cat Ing 
"Preferred'' 


Preferred, but not presently influential i 






..■ ■■ 


Evidence of Student Learning In Coursei 


> n 


1 


m 


Effort to Remain Current In Discipline 




3 


. 371 


Formal Colleague Appraisals/Teaching 


> 11 


k 


321 


Preferred^ but not Influential enough: 








Preferred and Influential 








Obtaining Advance Degree 


1 


2 


m 


Presently Influential, but not preferred 
by 301 or more: 








Evaluation by Department Chairman 


2 






Eyaluatlon by School/Department Comml ttee 


3 






Student Ratings 


k 






Service on Departmental Committees 


5 






Personality Traits and Attitudes 


6 






Time In Academic Rank 


7 . 






Publtcatlons 


8.5^ 






Informal Colleague Appraisals/Teaching 


8.5 . 






Service on Institution/System Committees 


10 
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Table 9 



A Cofnoarlson of Presentlv Influantial 
for Professional Schools: Col 


and Preferred 
lege Settings 


Factors 




Grouping 


Present Preferred 
Rank Rank 


Percent 
Indicating 
"Preferred" 


Preferred, but not presently influentfan 








Effort to Remain Current in Discipline 


> n 


1 


kk% 


Innovative Effort in Teaching 




2 


39% 


Evidence of Student Learning in Courses, 


^11 


it 


35* 


Preferred, but not Influential enough: 








Preferred and influential i 








Evaluation by Department Chairman 


2 


k 


35% 


Personality Traits and Attitudes 


k 


k 


35% 


Student Ratings 




5 


331 


Presently influential, but not preferred 
by 3Q% or more* 








Obtaining Advance Degree 


1 






Time in Academic Rank 


3 






Informal Colleague Appraisals/Teaching 


6 






Support of Department Policy and Goals 


7 






Service on Departmental Committees 


8 






Availability to Students 


9 






Evaluation by School/Department Committee 


10 
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Table 10 

A Comparison of Presently Influential and Preferred Factors 
for Physical/Natural Sciences^ University Settings 

(N=89) 



Percent 

Grouping Present Preferred Indicating 

Rank Rank "Preferred'' 



Preferred^ but not presently influentjal . 

Formal Colleague Appraisals/Teaching i 1' 3 37% 

Preferred^ but not influential enough : _ . , _ 

= Student Ratings , 7*5 ^ 35% 

Formal Col league Appraisals/Research 9 2- k2% 

Preferred and influential' 



Publications 11 781 

Formal Peer Appraisals (from off campus) 5 5 3^% 

Presently influential, but not preferred 
by 301 or more : 

Grant Support for Research 2 

Evaluation by Department Chairman 3 

Evaluation by School /Department Committee 4 

Time in Academic Rank 6 

Informal Colleague Appraisals/Teaching 7-5 

Effort to Remain Current in Discipline 10 
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Table 11 

A Comparison of Presently Influential and Preferred Factors 
for Social Sciences: University Settings 
(N=70) 











Percent 


Group 1 ng 


Present 


Preferred 


Indicating 




Rank 


Rank 


*'Preferred^» 



Preferred, but not presently influential 

Preferred, but not influential en oi 

Preferred and inf luentjaX - .^^ ^ . . 

PubUcationi 11^ 661 

Student Ratings k 2 301 

Presently Influential, but noj^ preferred ' 
by 3Ql or more : 

Evaluation by School/Department Committee 2 

Evaluation by Department Chairman 3 

Time In Academic Rank 5 

Public Presentations of Products 6 

Formal Colleague Appraisals/Teaching 7 

Obtaining Advance Degree 8 

Informal Col 1 eague Appra i sa 1 s/Research 9 

Informal Colleague Appraisals/Teaching 10 
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Table 12 

A Comparison of Presently Influential and Preferred Factors 
for Arts & Humanities^ University Settings 
(N-76) 



Percent 

Grouping Present Preferred Indicating 

_____ _ Rank Rank ^'Preferred" 



Preferred, but not presently influential i 

Preferred^ but not Influential enough ; 

Formal Peer Appraisals (from off campus) 7 2 331 

Formal Collaagge Appraisals/Research 9 3,5 321 

i 

Preferred and Influential; 



Publications 1 1 621 

Student Ratings 2 3,5 321 

Presently influential^ but not preferred 
by 301 or' more : 

Evaluation by Department Chairman 3 

Evaluation by School /Department Committee k 

Public Presentations of Products 5*5 

Time in Acadamlc Rank 5*5 

Service on Departmental Committees 9 

Obtaining Advance Degree 9 
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Table I3 

A Comparrson of Presently Influential and Preferred Factors 
for Professional Schools^ Unfversfty Settings 



Percent 

Grouping Present Preferred Indfcating 
■ Rank Rank "Preferred"' 



Preferred, but not presently influential : 

Preferred, but not influential enough i 

Preferred and influential : 

Publ I cat ions 1 2 39| 

Student Ratings 3 1 57I 

Presently influential, but not preferred 
by 30% or more : 

Evaluation by Department Chairman 2 

Public Presentations of Products k 

Evaluation by School /Department Committee 5 

Obtaining Advance Degree 6 

Service (no fee) to Community 7 

Time in Academic Rank 8 

Elected Offices in Organizations 9,5 

Service on Departmental Committees 9,5 
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Interview _Da^a_ 

Two sets of Interview data are available for analysis, Campus presidents 
and vice-presidents were Individually interviewed during year one, along with 
either members of the campus^wlde promot Ion/ tenure review committee, 
if sueh a committee eKisted, or upper level administrators, such as deans 
or division heads. The second series of Interviews were held during year three 
with administrators who had had the rrost contact with the projecti and they will 
be mainly discussed under ''impacts^' (Both sets of Interview questions are shown 
in Appendices A and C*) 

The initial interview sought information regarding the decision making 
process j i.e., on what basis are choices finally made regarding the award of 
tenure or proration, A number of generalizations can be made about these 
interview data. 

Whether i t be a campus president or a campus'^wide committee, heavy reliance 
has to be placed on the candidate and his/her colleagues to collect and submit 
evidence warranting a favorable decis ion , The most common complaints by the 
decision makers were about the quality and quantity of the evidence reaching them. 
Decision makers repeatedly cited how evidence derived In unsystematic ways were 
submitted, often also lacking any clear qualitative evaluations. They most 
frequently pointed to inadequate evidence In support of teachings and although the 
use of student ratings was encouraging to nearly all, they said the latter suffer 
badly from the wide variations, even within a department^ in the questions 
asked and the interpretations made. The decision makers appealed for some 
comparative (normative) basis upon which they could render a judgment* rather 
than only upon each individual's unique data. 

The evidence submitted on one's research was generally less troublesome, 
in an Immediate sense. Journal articles and books are tangible products which 
can generate Judgments, even by Judges outside of the content area. The use of 
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such tangible evidence became a means by which the decision mal<ers could 
justify a fairly uniform criterion, especially on large campuses where the 
probability was high that no decision mal<er would be directly l<nowledgable 
about the candidate. (This is, of course, just the opposite case on the small 
college campuses, where often presidents, and certainly review committee 
members, would have some direct Icnowledge of the candidate.) 

The ease with which publications can serve as useful evidence also posed 
a dilemma for many of the decision mal<ers, who sensed that they felt forced to 
place disproportionate weight on publishing, by default', i.e., no equally 
substantive evidence was conmonly available to Judge teaching or service. A 
not uncommon appeal, then, was for more effort to be placed on improving the 
amount and quality of evidence on teaching. Many interviewees, especially 
at the presidential level, also voiced concern about the tendency for faculty 
to eschew making substantive recommendations for denial of promotion/tenure. 
Instead, such hard decisions were "bucked to the' top." There seemed to be 
an underlying tone of Impotence expressed by the administrators In getting 
faculty to more responsibly assure this Judgment burden. 

In addition, the heavy reliance in universities on using off-campus sources 
,,for evaluative judgments of a candidate's research is in Jeopardy. These 
sources have apparently been drastically reduced because of their reluctance 
to pass Judgments which can then be challenged by candidates (due to open records 
laws). The latter is an ironical twist, for the validity of such evidence ^ 
has apparently rested very heavily on its unchal lengabl I i ty rather than any 
inherent, publicly verifiable worth. Many of our interviewees were not hesitant 
to characterize these outside letters as frequently written by persons who fall 
to distinguish between "an evaluation and a testimonial." It Is this latter 
feature that has particularly suggested to the authors the disingenuous aspects 
of such present forms of evidence when used to evaluate faculty applying for 



F»1 



promotion or tenure. We certainly feel such outside review iources have 
potential, but not without a coniiderable revision In the p'^oceduras for 
collecting the data* As a minimum, explicit efforts have to be taken to 
assure adequate mot i vat ion by the reviewer and clarity of his/her purpose 
In doing the review* 



Project Impacts 

As mentioned in the Methodology section of this report, the project 

staff were to seek evidence of intended and unintended impacts which could 

be attributed to the three year study. Five potentially favorable types of 

influence were enumerated at the proJect^s inception^ 

I. The anxjunt and kinds of data collected to assess the 

quality of faculty performance^ especially as they relate 
to the teaching function, 

II* The anrount of specification and utilization of procedures 
for assessing faculty performance to make tenure and 
promotion decisions. 

Ill« The types of differential weightings given to faculty 
functions in assessing Individual performance. 

IV. The amount, types and utilization of professional 

development activities, especially as these relate to 
the teaching function* 

V* The perceptions of students, faculty, and administrators 
regarding the calibre of the education experience provided* 

Evaluation effor.ts were definitely to mclude data on types I^II^S III, and on 
IV and V If possible. We were not successful In collecting any data on number V, 

Three kinds of data were collected during the final few months of the project 
The first, and most representative, were from the faculty who completed the FPQ 
in 1976» The second were from admin Istrators and promotion committee members 
who were interviewed by project staff. Though th^ese Individuals were always 
Informative and more substantive in their responses than the FPQ respondents, 
the interview method Introduces bias which must be acknowledged* We sought out 
persons on each campus whom we either knew, or strongly expected, would be 
familiar with our activities and reports. Not all those familiar with the project 
gave favorable assessment, and we shall try to note these. Nor were all even 
familiar with the project, much to our chagrin. Finally, the third kind of data 
refers to anecdotes, unsolicited comments, and activities begun as by-products 
of the project. 
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Impact h" The amount and kinds of data collected to assess the quality 
of faculty performance, especially as they relate to the teaching function . 

The tremendous increase, since 1973, in the use of student ratings of classroom 

performance was by far the rripst comprehensive Influence which could be partially 

attributed to the project. In 1973 only a small percentage of OSSHE college and 

university departments were regularly sol icf ting student ratings on their 

instructors, and an even smaller percentage of the ratings were actually reaching 

promot ion/ tenure comml ttees , However, every campus in 1976 was utilizing student 

rating data, with a marked increase also in the quality of such data. The latter 

probably helps account for the frequency with which faculty nominated such 

evidence when asked for preferences. 

The FPU data relevant to the use of evidence from students can be seen in 
Table ]k. The question did not speak to classroom ratings only, but all of our 
supportive data have confirmed that other form of student input have been rare. 
We can see that a majority of all faculty perceived an increase in use since 
1973. Individual campuses ranged from a 381 increase (college B) to an 891 
increase (college A)* Both of these campuses had used quite limited student input 
prior to the project, but both sought assistance from the project staff In 
deveJoR^ing suitable rating forms and procedures and each subsequently implemented 
comprehensive use of them. The final difference may largely be attributed to 
the presidents^ one strongly endorses this form of evidence for promotion/tenure 
decisions, while the other openly disparages it. 

The high perception of increased use iit university C can also be ascribed 
in part to the project. On this campus tha project staff worked closely i (a) with 
the associated students in developing a campus-wide form; (b) with numerous depart- 
ment heads and school deans In selecting appropriate rating forms; and (c) with 
proniot ion/ tenu re review committee members, providing them with our Interim 
findings and thus encouraging thm to solicit such student input in their reviews 
of teaching adequacy- 
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Tables 15 and 16 further address this project impact, in showing substantial 

increases in the collection of systematic evidence to verify the adequacy of a 

candidate's instruction and research. These faculty perceptions are very 

reassuring, though each campus' promotion process is still unduly dependent upon 

some questionable forms of evidence, Unfortunatelyj upsurges in collecting more 

systematic and formal evidence may be creating, on some OSSHE campuses, unintended 

consequences J which one administrator described as followsi 

Increased emphasis on the importance of instruction has been 
accompanied by increased frustrationj because of the fact that 
the means for collecting the data are not well understood. 
Other criteria^ such as degrees obtainedp years in rank, and 
prior professional experiencej are still emphasized in a 
relatively straightforward fashion. 

Perhaps equatly distressing is a situation such as the following, where 

even after distributing our report on one campus and discussing it with 

administrators, one department head answered the following question in this way: 

What changes have you seen In the relative importance of promotion/tenure evidence 

pertaining to the adequacy oF a candidate's Instruction? 

Course syllabi come in to me on a yearly basis. They're stored 
in my office. There are also a lot of other informal things 
I try to keep track of, e.g., faculty availability is judged by 
me as I wander through the halls--- 1 do this for one hour each 
day. Promptness of task completion Is another area of concern 
which I have when I evaluate the performance of faculty. All 
in all j I believe that I deal with evaluation In an open and 
cons i derate fash ion , 

We would like to claim that such abysmal ignorance of sound evaluation procedures 
i s rare, but it Is not. 

Impact II: The amount of specification and utilization of procedures for 
assessing faculty performance to make tenure and promotion decisions . Any evidence 

of project Influence on changes In this area, beyond what can be inferred from 

data already cltedj comes mainly from Interviews and anecdotes. The most 

substantive change came from an entire university school faculty, which completely 

rewrote Its policy and guideline document on promotion and tenure following our 

report to them. The dean was determined to reniove as much ambiguity as possible. 
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once our report had confirmed what he and others had long suspected. In other 
cases one campys revised its guidelines and instituted a campus-wide review 
cormilttee, while another implemented an entirely new set of criteria and proce- 
dures for yearly review of al I faculty. 

Sometimes our attention has been focused on campus^wlde committees, and a 
member of one of these responded as follows to a question about criteria clarity 

The Cormiittee has tried to increase clarity but 1 don't think 
that we have been very successful. We will continue to work In 
this area. Clarity of evaluation criteria Is largely a departmental 
respons i b I 1 i ty • The Committee expects certain departments to be 
leaders in the area and 1^ for one* am disappointed that some do 
not exhibit the kind of leadership that Is going to be required If 
departments are going to make significant gains in clarification 
of evaluative criteria* For example^ the colleges of education 
s^hould be leaders in the area of the evaluation of teaching* 
Typical ly* they are not* 

It IS certainly reasonable to assume that there will be many more changes 
In the clarity of campus criteria and procedures^ once the present report has 
had time to be considered by OSSHE faculty and administrators* The impacts 
from our i ndlvldual campus analyses sometimes took twelve months to emerge. 
A few quotations from our intervlavs may help the reader see some of the 
problems in stimulating Institutional change* 

On one campus a dean responded to the question of our campus analysis 
having had any influence as follows: 

I think that our Campus Analysis has had three elements of 
Influence, First, project staff provided a resource capability 
to Senate committees In their consideration of policy proposals* 
Second, the Campus Analysis (with Its presentation of preferred 
and Influential factors) gave direction for the development of 
both criteria and weighting procedures and, in part, Justified 
the final system as It Is now developing* Finally, the periodic 
visits by project staff focused attention and occasioned the 
reconsideration of administrative thinking relative to process 
and evidence used to evaluate faculty* 

Unfortunately, a department chairman on the same campus said: 

No. I don't think the Campus Analysis did have much Influence 
on the process or the evidence used to evaluate faculty. The 
reason is that people like myself just discarded It because of 
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other complications and because of a lack of interest in, 
the Issues it raised. More recently this lack of interest 
has subsided somewhat and I believe that the Campus Analysis 
Is beginning to be somewhat related to present administrative 
considerations of promotion/tenure process and avidence. 

On another campus three responses are Informative. A department head 

said: 

The Influences were not as much immediate as they were long- 
range. Things are happening now as a result of that report 
and could well be attributed to it* It was able to force 
people to re-examine a number of criteria and the way in which 
these were used, and it looks like some new statements will be 
made on this matter. I feel a number of new alternatives should 
emerga and the benefit from this was that It got everyone 
thinking about the whole process, 

A second department head replied as follows, 

I think that because of the report there has been a great 
deal more questioning and concern by faculty about the 
whole process of promotion and tenure decisions. Faculty 
now seem to want more voice in what is going on^ and it has 
stimulated concern that is carrying on into the present in 
revising some of the criteria and process. 

Finally, their dean said. 

Only in this last year has the report become widely read. 
We are just beginning to see Its impact. The Faculty 
Senate's present efforts aimed at revision of the criteria 
is a direct outgrowth of our Campus Analysis and its findings. 
The report's central flndingp as far as this group Is con- 
cerned, is the fact that faculty members viewed the promotion/ 
tenure criteria differently than their administrative 
counterparts . 

Another campus dean gave the following balanced perspective on our efforts 
to stimulate change amongst faculty^ 

I believe the Campus Analysis had little direct Impact on 
process or evidence used to evaluate faculty. This needs to 
be qualified somewhat. In the beginning, Its findings 
were viewed as so patently reliable that the general thrust of 
its recommendations has become more and more undeniable to 
faculty. Over time, faculty have begun to view the Analysis 
and its findings and recommendations as definite Indicators of 
issues which need their attention. More recently, faculty 
attention has begun to focus more and more Intensively on these 
Issues, and I believe over time that significant action will 
occur. It is important to be realistic In matters related to 
promotion and tenure by recognizing that significant change will 
probably not occur over a short number of years. It can take 
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anywhere from five to ten years to make a significant Irnpact 
on proGedures and perhaps even longer on evidence. However, I 
do not believe thafc this necessitates a negative conclusion 
regarding the actual jmpact of your Campus Analysis. 

From our own point of view, those campuses which attributed the most 

influence to project efforts were clearly from our college group. They were 

small enough in size, and perhaps limited enough in mission, to enable discussion 

and interchange amongst faculty, administrators, and project staff. Discussions 

were seldom effected at our three universities, where their greater size and 

multiple missions made campus-wide dialogs formidable. Nevertheless, our 

results do suggest that heightened awareness of shortcomings in faculty 

evaluation, especially regarding teaching, has also occurred on our university 

campuses , 

Impact III: The types of differential weightings given to faculty functions 
in assessing individual performance . This was a particularly difficult Impact to 

try to assess. When It was formulated early in the project, we had been led to 
believe that (a) policy statements about the importance of Instruction, e,g, , 
"more weight", "the most attention to.,,", or only "superior teaching", etc, 
could be taken literally, and (b) evidence would be In dossiers as well as in 
minutes from review committees. Neither assumption was verifiable. Campus 
policies on confidentiality of records prevented our staff from actually 
examining dossiers, although administrators usually shared sample ones with 
us. Furthermore, several faculty and department heads gently suggested that 
we were quite naive to think that the evidential sources for promotion/tenure 
decisions would be clear from either connilttee minutes or actual dossiers. 
We still do not knw. 

One instance of change on a large campus was encouraging to us. We had 
interviewed each campus-wide committee member and then shared their responses 
(anonymously) with them. Several seemed shocked that such contradictory and 
poorly defined criteria were being used by the members, and subsequent efforts 

,58 



were taken by the chairnan to generate more consensus on criteria. A year 

later one comfnittee member said the relative importance of teaching, research, 

and service was as follows^ 

The Promotion Tenure Comriittee takes the position that 
Instruction rates equally with scholarship (scholarship^^ 
40l* instruct lon--^OI, Institutional service--10l, and 
community service-^lOl) * 

It's hard to say whether or not this forma II zed approach 

has resulted In an increased emphas i.s on teaching. There is 

no doubti however, that the emphasis on Instruct iont as far as the 

Committee is concerned, is much more clear now than It has been in 

the past, 

A college dean responded to the same question In this way. 

I have corre to recognize the need to be more tangible when 
evaluating Instructioni I came to this realization largely 
as a result of the AFP study, I look for sources of evidence 
that vn 1 1 support decisions about the teach mg function. At 
the same time, however, one must keep In mind that academic 
preparation remains an essential quality of sound instruction. 
This was also true In the past. The difference now Is that I am 
attempting not to take my faculty members' teaching for 
granted. 

All other results are anecdotal and neither more nor less convincing than 
these. It Is our impression that the relative weights given to the primary 



functions are probably not iubstant ial ly 
What has seemed to change though, is the 
and substantive evidence on Instructionj 
change If the quality of Instruction Is 



changed as a consequence of our activities 
Increased emphasis given to systematic 
which would be a very impontant 
ever to be verified or improved. 



Impact IV: The amount, types, and utilization of professional development 
activities, especially as these relate to the teaching function ^ The results 

regarding this impact were disparate, and attributing the positive ones to the 

project would be highly tenuous. A few campuses were able to save some money 

from operating budgets and devote the savings to special instructional efforts, 

niainly In the form of workshops or ml n I --grants * These efforts are commendable 

and probably will be repeated whenever possible, but they could, hardly be 

attributed to project activities. 
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Two major instructional development projects were launched in the fall 
of 19761 3nd these were direct outgrowths from the project. One is supported 
by State funds at Eastern Oregon State College, and the second by FIPSE funds 
Portland State University, Each is devoted to assisting faculty In the 
improvement of their instruction, and in collecting credible evidence on how 
such improvements affect student learning* We fee] confident that neither 
campus would have similar efforts presently underway had it not been for the 
present project* 
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Table \k 

Responses to ilueitlon I; "Since I§j3 what changeSf if my, have 
yoij seen in the use of proinotion/tinyre evidence 
collected from students regarding adequacy 
of instruction?" 



Percent Indicating Each Reiponse 



Unit Responding 


A decrease 


No change from 


No change from 


Increase 


Don'Y" 






in use 


low usage in past 

— - r 


high mm in past 


in ysi 


Know 




College Faculty Combined {\^\kk', 




281 


m 


521 


51 


\ 


Collage A (N«27) 


0 


11 


0 


89 


0 


M\ 

j 


College B (fH3) 


3 




3 


38 


10 




College C (N«5a) 


, 7 


20 


20 


^9 


2 




llniversity Faculty Cooibined 




191 


lOl 




n 




University A (N-1^9) 


6 


15 


15 


62 


1 




yniversity B {HI) 


/ 


30 


9 


51 


I 




University C (H?) 


3 ■ 


13 


k 




3 





Table 15 

Responses to Question 2a: "Since 19J3 what changeSs If any, have 
you seen in the importince of systematic ividenee ' 
used to verify the adequacy of a promotion/ 
tenuri candidateis instruction?'' 



Unit Responding 



Percent Indiciting Each (Response 



A decreise In Low liiiportance in High importance in Increased " Don't 
importance past and no change past and no change importance i( now 



College Faculty Conibined {W^lkk) kl 

College A (N^2?) 7 

College B (IH9) 2 

College C M) 5 



m 
15 



15 
15 

20 



56 k 
28. 15 
^1 3 



81 



University Faeulty' Combined {\\~]}k] 3| 



ynlversity A [M^] 
University B [mM) 
University C (H/) 



5 



191 



31 



211 
Ik 
18 
19 



511 
51 

III 
58 



63 
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Table 16 



Responses to tetion 2b: "Since 19?3 what changes, if anyi have 
you seen in the Importance of systematic evidence 
used to verify the adequacy of a proniQtion/ 

tenure candidate's Research or ^ 
Scholarly Development?'' 



Unit Responding 



College Faculty Cofiibinad (N=lW) 
College A {M]] 
Col lege B (Hj) 
College C (Hi) 



5 
10 



Percent Indicating Each Response 



A decrease in Low importance In HigF importance in Increased Don't 
Importance past and no change past and no change _ importance Know 



m 

22 

35 

kk 



m 
1 

20 

Ik 



251 
56 

■ 

20 

15 



101 

7 

13 
3 



University Faculty Combined [l^'^jk) k% 



University A {mks) 
University B (|HB) 
University C (n-3/) 



1 



51 



10 
k 



m 

3? 



371 

32 
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OVERVIEW AND IMPLICATIONS FOR ACTION 

The present findings suggest some new ways to consider the policies and 
procedures guiding the evaluation of faculty. The popular claim that faculty 
on university campuses ignore their teaching responsibilities, in contrast 
to faculty at small college who do not, remains much too simplistic. In 
fact, we found that the university faculty have made somewhat greater efforts 
to collect observable evidence regarding their teaching than have the college 
faculty, when promot ion/tenure decis ions were being made. But the overwhelming 
conclusion from this study is that there has been an absence of systematically 
collected evidence to evaluate instruction on both kinds of campuses* 

The evaluation of college and university faculty has always been complex^ 

Just as the evaluation of education has been, and the difficulties are present 

on every campus. Yet, the university faculty, who have research responsibilities, 

may have some advantages, since tangible products from research and/or artistic 

activities are easily obtainable and visibly rewarded (Tuckman and Hagemann, 1976), 

We can conveniently get Judges to assess paintings or a symphony or a book of 

poems or a research experiment* One can get total agreement on whether research 

grants are obtained. But what procedures conveniently exist for Judges to 

determine whether students learned, or whether a course's content is obsolete? 

Faculty In both colleges and universities do have instructional products to 

show, but the absence of credible evaluation procedures means that neither usually 

produce any observable evidence for performance evaluations. This dearth of 

systematic evidence used to evaluate the teaching function Is a finding Astin 

and Lee bemoaned a decade ago. 

If the ultimate measure of the teacher's effectiveness is 
his impact on the student-=a view which few educators would 
dlspute^^it Is unfortuante that those sources of information 
most likely to yield Information about this Influence are 
least likely to be used (1966, p, 364), 
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In our intervi©^s with campus executive officers, we often found them 
deeply aware of this lack of evidence regarding teaching. Their degrees of 
discomfort over this ranged from resignation to sharp criticism of department'' 
level administrators for tolerating such practices, The university admlnlstra^ 
tors more often expressed dissatisfaction with the quality and quantity of dossier 
materials than did the college administrators, but this is hard to evaluate. 
The latter, for obvious reasons^ had much more confidence in their knowledge 
gained from personal acquaintance with the candidates. We view such acquaintance" 
ships as deceptively misleading in Judging quality of Job performance. They 
might inform a president about the quality of a faculty member's mind, or 
through chance observations the diligence shown In being present on the Job. 
But, insofar as performing the instructional functionp we submit that other kinds 
of evidence are needed and they are typically neither sought nor supplied. 

The present findings, as a whole, point to significant gaps between 
institutional policy statements regarding promotion/ tenure criteria and the 
data collected to verify their attainment* Such findings are not unique tb 
Oregon* Other research (e»g., Eble,' 1972) covering the last decade has shown 
faculty members to be unable to generate much substantive progress in developing 
equitable and adequate methods for assessing performance, whether of faculty, 
programs, or Institutions, The reality Is that faculty members remain In 
control of what Is valuedj and the determination of what evidence will be used to 
substantiate the attainment of those values* Campus administrators are commonly 
torn betv/een this expression of the idealism of the academy on the one hand, and 
the management realities they must face, including the pressures of Judging 
one's colleagues and their work, on the other. The consequence Is frequently 
a stalenat^j regarding proposals for change In the assessment process. However, 
the present national economic forecasts may exert a new kind of pressure 
for changes amongst academics. Campuses are unlikely to be growing In the 
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foreseeable future, and therefore harsh constraints shall bear upon idministra- 
tors to assure high quality performance with eKisting resources. The times, for 
better or wor se, -may be optimal for ra I s ing fundamental quest Ions about the 
purposes and methods of evaluating faculty. 

In the short span of our study we have seen some not^orthy efforts 
within OSSHE to improve faculty performance review procedures* Almost without 
exception, the impetus for these efforts originated from highly placed adminis" 
tratorSj an observation clearly in support of our bias that direction for improve^ 
ment must come from the top* This is not to say that we advocate unilateral and 
arbitrary imposition of changes from above. To the contrary, we firmly believe 
faculty members must be active In the design and development of the procedures. 
Further, our observations within OSSHE, and on selected campuses across the nation 
confirm the ability of many administrators and faculty to work together productive 
ly when administrative leadership is present* 

So where, on the basis of the present findings, might a campus administrator 
begin to develop incentive models capable of positively Influencing the quality 
of instruction? We would recommend a methodology similar to the one used in this 
project. This would mean two kinds of focused actlvltias would be initiated. ' 

ihe first would be to find a means^^an Individual, a committee, a task 
force""for reviewing and thoroughly analyzing the Institution's policy and guide"" 
line statements governing salary, tenure and promotion decisions, to establish: 

a) their compliance with administrative rules; 

b) that the emphasis given to the three major professorial functions, 
vii*, instruction, scholarship, and service. Is consistent with the 
established mission of the Institution; 

c) an optimal degree of consistency between policy and guideline documents; 

d) the extent to v^hlch criteria and their minimum necessary evidence, as 
required for faculty and administrator evaluation, are specified; and 

e) the extent to which evidence to be gathered by systematic methods Is 
made available to the duels Ion makers. 
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The second set of activities should focus upon faculty perceptions (and 
preferences) ragardi ng what influences salary, tenure^ and promotion decisions. 
The discrepancies between policy intentions and what is perceived, and there will 
be many, are not only inforTOtlve but also excellent subject matter for dialogi 
and the FPQ instrument could serve as a good way to determine such discrepancies. 
This data could then be compared with the policy analysis* and steps taken to 
alter, increase and/or eliminate types and sources of evidence. 

It is our expectation that any campus completing these two sets of . act ivl ties 
will be in a position to simplify and improve their faculty evaluation procedures* 
and thereby provide considerably more realistic Incentives for faculty, For 
example, there should be no way to continue the justification of a "shot gun^' 
approach to assessment. The business of collecting any and all kinds of data 
having some face validity can stop. Our findings have repeatedly shown that 
a very limited number of criteria and review sources should satisfy (a) the 
reality of making decisions (decision makers do not uti 1 i ze dozens of variables 
In making a choice, even though they may think they do), and (b) the preferences 
of faculty, who have said In the study that they would prefer to be Judged with 
systematiG, product^oriented evidence. 

A good start on the simplification of the review process would be to affirm 
the use of those criteria and review sources in which there Is consensus that 
they are (a) definitely Influential and (b) cons istent wi th institutional policy, 
(The criteria shown In Figures 7 and 8 would be an example of the first half 
oF the conditional statement.) Such an affirmation process should produce 
a list of criteria quite reduced from Its original size. 

Furthermore ^ the process would serve to eliminate all criteria which 
occupy an ambiguous status, Table 5 Is a good example of how numerous criteria 
allegedly influencing promotion doclslons can be ambiguous and provlda only noise 
the decision process. These should be removed from guideline documents until 
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consensus can be achieved regarding their valuer and operational definitions 
assigned to them. Just this step, in the Oregon System, could save inestimable 
amounts of energy, and probably reduce widespread faculty cynaclsm about the 
present promotion process. And what good does it do the credibility of a 
campus to list a criterion, e,g,, ''formal col league appraisal of one's teaching*' 
IS required for favorable review, when neither formal procedures nor eKplicit 
criteria actually exist? (These actions might prove devastating for the usually 
unwritten criterion "evidence on personality traits and general att i tude"--but 
all to the good* If it cannot be operationally defined, then why should It 
be allowed to exert influance on a salary, tenure or promotion decision?) 

We would anticipate that somewhere between four and six criteria will 
initially emerge and be justified for use on a college or university campus, 
following a comprehensive review of the piorTratron process. These would definitely 
include (a) student classroom evaluations, (b) publications and/or artistic 
products, and (c) evidence of student learning* Two review sources are likely* 
by department chairpersons and either department or school committees , Reviews 
by Interdisciplinary groups, e,g*, a un I vers I ty^w i de committee, raised serious 
questions for us because of the wide disciplinary differences in criteria and 
standards. Rather than force a dilution by merging, we would prefer to see 
broad campus disciplines set their own criteria. Such differences, according 
to our data, are somewhat less likely to be a problem on college campuses, 
where a campus-wide set of criteria might not be impractical to develop* 

There Is some probability two or three other sources of evidence will emerge, 
perhaps In the "service to the campus" area, but no n^re will be necessary nor 
probably defendable from a measurement viewpoint. Faculty and administrators can 
then re-define their purposes and the incentive process necessary for promoting 
and maintaining high quality per fontance , ^ Teke, for example, the significant 
influence given to the attainment of a doctorate on the college campuses* 
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It was explained to us that this is the best indication of a person's, currency 
in discipline. Why is this so, and for how long would this be true-*'"one year, 
five years, twenty? Universities do not rely so heavily on such a proxy. Their 
position essentially says tangible research products must be produced and Judged 
by one's peers. The doctorate Is considered an excel lent^ but not absolutely 

necessary, way to acquire the requisite skills and knowledge to produce research 

/ ^ . \ ' " ■' ■■■ 

p^oducts, It is a means to an end. On some'^co'nege campuses it may have 

become an end, in itself. 

What this thorough review of the promotion process could accomplish^ in partt 
is a visibility for the instructional function that has not been present on the 
campuses. In order to serve several major audiences or consumers, faculty are 
typically hired to perform three major functions. Research/scholarly development 
mainly serves society and academic disciplines. Instruction mainly serves 
students. Service activities mainly serve the community and the campus. The 
evidence toll ec ted to evaluate a professor's performance should reflect these 
audiences^ but our findings demonstrate that the student audience do^r.not get 
the same degree of concern as the other audiences do. Concerns about the quality 
of instruction were not found to absorb significant administrator or faculty 
energy. Of course good instruction is desired, but when there Is no formal 
system of assessment i t runs the danger of becoming taken for grantadj and admlnis 
trator energy gets focused only on the negative extremes gross negligence of 
teaching responsibilities. Genuine and frequent attention by administrators to 
teach Ing, ' a long with clear means of rewarding high quality performance, should 
markedly legitimatize a function that has for too long been given benign neglect. 

A large proportion of our faculty were found to produce high quality 
performance on all our campuses. Likewise, we saw excellent leadership In 
many campus administrators. With these elements present, and we believe them 
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to be on any campus, continued improvement is possible, and the findingi 
of this study can support those policy makers who desire to more closely 
match policy statements with actual practice. 
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1973 Interview Form Used with Presidents, 
Deans, and 01 vis Ion Heads 
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Interview Form for Institution Administrators 
Relating to Tenure-Promotion 



Name " Data 

School __________ Oept/Dlv Rank 

1, From whom do you receive tenure and proration recommendations? 



(a) In what form do you receive them? 



2. What do you see as the main perspective or focus for each iource 
named in Question 1? _ 



3* To whom do you make tenure and promotion recommehdat ions? 



(a) In what form do you transmit them? 



is the final decision an Individual or collective one? 
Individual Collective 

(a) If a collective decliion* was consensui eitabllshed on the 
meaning of rank and/or tenure? Yes No 
(la) If yeip what was It? _________________ 



Continued 

(b) If a collective decision^ was there any effort to set standards 

for tenure and/or promotion? Yes No ____ 

^^^^r (lb) If yes, obtain copies or a deseription 



What kinds of evidence were most convincing to you in Judging the 
functions of: ^ 
(a) Teaching ' 



(la) Did you make any distlnctloni in this evidence when Judging 
assistant to associate or associate to professorr If so^ 
what were they? 



(b) Researoh 



(lb) Did you make any distinctions In this evidence when Judging 
assistant to associate or associate to professor? If so> 
what were they? . 
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(c) Institutional Service 



(le) Did you make any distinctions !n this evidence when Judging 
assistant to associate or associate to professor? If so, 
what were they? . 



(d) Community Service 



(id) Did you make any dlstinctioni in this evidence when Judging 
assistant to associate or associate to professor? If so, 
what were they? ■ 



What gaps or problems occurred In the data provided In the candidates' 
files? - 



(a) How were these usually resolved? 



(b) Any recommondat ions? 
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7. What kind of cases are the most difficult for you to decide? 



8, How do you handle differihg standards between departments (e.g*^ some 
rate everyone "eKcellent" while others use less superlative descrip" 
tions^ some supply mainly "soft" dataj etc.)? ' 



g. What changes ^ if any, would you like to see in the review process? 



10. What advice would you givet in light of your experience, to a new 
assistant professor to assure! the attainment of promotions and 
tenure? __ _ __ _ ____ __ ______ 



I.. J 



AFP 5/7^ 
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Appendix B 
Sampling Hethodologyi 1973'76 

This section presents (a) the sampling philosophy that guided data collec- 
tion activities^ (b) departmental sampling procedures employed during the 
73"7^ academic year, (c) system^^wide sampling procedures followed during the 

academic year^ (d) sampling of professional schools during the 7^"75 
academic year, and (e) the 75'76 follow-up of 73~7^ respondents, 
Oyera n Samp 1 i ng Ph i 1 gsophy 

The sampling methodology described in this section was designed to culminate 
in sequential data collections that would enable project staff to meet data 
requirements as set forth in the Novemberp 1973, AFP Evaluation Plan* In each 
case the purpose of sample selection activities was to select sites so that 
generalizations could be made which, in turn, would reflect upon each of five' 
broad research questions (referred to as "Impact Areas" in the Evaluation Plan). . 

Included amoung the five research questions were* (a) What is the amount and 
quality of evidence used to collect and to assess faculty performance; (b) what 
procedures are used for collecting evidence of teaching quality and how is the 
evidence utilized in the tenure and promotion decis lon^maki ng processes; 
(c) are there differences in the weights assigned or ascribed to teaching, 
professional developrrent, and service when a decision Is made regarding a faculty 

member's promotion or tenure status; (d) what effect wl 1 I the presence of AFP - — 

i _ 

Staff within departments or within DSSHE have upon the amount and quality of 
faculty professional development activities that have as their primary purpose 
the Improvement of teaching quality, and (e) what are the perceptions of students 
regarding the calibre of the educational experience they receive? 

To help address these questions it was decided to begin in the 73^7^ academic 
year by collecting Faculty Interview Form (FIF) data from two stratified 
"samples" of OSSHE faculty members. This process resulted in the development 

8X 



of- a departmental simple (selected by department for adequate di scip! Ine and 
instltucjon coverage) and a system-'wide sample (selected by individual for 
adequate OSSHE Goverage). 

In the 74^75 academic year the FIF was again used as the primary data 
collection instrument for assessing professional school faculty member's percap- 
tlons of the promot ion/ tenure process* This sample included faculty from four , 
professional schools at the University of Oregon and from the Health Sciences 
Center - 

The nature of each of these three samples and the procedures used to 
select thm are described in the following three sections^ 

Individuals within the departmental, the system^wide, and the professional 
school samples were selected through the use of identical sample selection criteria 
In order for a faculty member to have been included in either one of these sample 
groups, he must have been at least at the rank of assistant professor with a 
*50 PTE appointment in an instructronal area* Both of these criteria were 
examined for allSystem faculty by using the fall, 1973* OSSHE Payroll List, 

The reader will no doubt notice that the sampling approaches we have chosen 
to use deviate somewhat from standard sample selection criteria* To the extent 
that procedures do meet these criteria, we should be able to make a strong case 
in favor of the argument that whatever factors 1 other than the project j influenced 
one sampling site will have influenced other sites either equally or, when obvious, 
vn 1 1 be accountable for i n our analyses. To the extent that our procedures do not 
meet these criteria it should be stated that we chose to select our sample In 
the manners described for a variety of reasons and stand ready to defend both 
our selection schemes and the integrity of the data which they generated. 
Departmental Sampling Procedures (73''7^) 

During the early fall, ig73i a four-category departmental classification 
was developed which, in essence, allowed AFP Staff to classify departments by 
discipline types* The categories included were* (a) fine arts and humanities; 
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(b) physical/natural sciences; (c) social sciences; and (d) professional 
schools. The fine arts and humanities category includes departments such as 
music, English, philosophy, humanities, and art. The phys i ca 1 /na tura 1 scjences 
category includes departments such as biology, zoology^ mathematics, and chemis'- 
try. The social sciences catagory includes departments such as history, 
psychology, economics, and anthropology. The professional schools category 
includes departments such as business, education, agriculture, and engineering. 

Each institution was classified according to the highest degree offered to 
its students. Institutions classified as PhD granting or "University'* types 
included Oregon State University, Portland State University, and the University 
of Oregon. Institutions classified as bachelor or master degree granting 
institutions or "College" types included Eastern Oregon StatWCoMege, ^Oregon 
College of Education, Southern Oregon State College, and Oregon Institute of 
Technology. The latter of these institutions was excluded from most data analyses 
because of its specialized nature. 

Using these two class i f- :at ion schemes (i.e., discipline type and Institution 
type) AFP Staff selected 25 departments which, in aggregrate, provided adequate 
coverage of each type of discipline within each type of institution* 

Twenty-three of the 25 target departments [32%) agreed to participate with 
us by providing data for the AFP Project. Within each of the 23 departments 
AFP Staff (a) conducted a personal interview with the faculty administrator 
(usually the department chairperson), (b) examined all written procedures and 
guidelines, and (c) constructed departmental information-process flow-charts of 
the departmental tenure/promotion *jctivities. 

The next step was to collect Faculty Interview Form (FIF) data on a 
voluntary basis within each department. Included within these 23 departments 
were a total of ^23 eligible faculty members ^.50 FTE assistant professors 
or above). Packets were developed for each of these faculty members* Each 
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packet contained a cover litter that explained the purposes of the FIF, 
directions for completing the FIF, the FIF itself, and a stamped se I f ^addressed 
return envelope. 

Before data collection began, the decision was made by AFP Staff to do 
everything that was reasonably possible^ within the limitations of the Project's 
resources, to attain departmental response rates of 701 or above. It was felt 
that the data which would result from response rates any lower than 701 would 
be too biased to have the degree of confidence that we felt was necessary for 
the kinds of generalizations and conclusions that we desired to have the 
capabiMty to make. 

Data collection began in late January of 197^ and continued through 
April 15i 197^. During this period of time, AFP Staff tried to personally 
deliver each faculty member's FIF packet in 20 of the 23 departments (in three 
of the departments it was decided^ for a variety of reasons, to mail the FIF 
packets to each faculty member^^this procedure resulted In an overall response 
rate for the three "mailed" departments of 66,01)* If, after a maximum of three 
visits to the 20 "personal contact" departments , an AFP Staff member was not able 
to personally contact a faculty member, the FIF packet was left with a handwritten 
note which expressed our regret that a personal contact was not made and requested 
the faculty member's cooperation. Of the eligible faculty members in the 20 
"personal contact" departments 302 or 85*51 of the FIF packets were personally 
delivered and accepted, A total of 37 or 10.5% were left with a personal note, 
and ]h or of the contacted faculty members refused to participate. This 

^.0% refusal rate was included In the calculation of the overall response rate* 
Subsequent to the delivery of the FIF, If no response was received from a faculty 
member within two or three weeks, then he received either (a) a follow-up 
"reminder letter," (b) a phone call from an AFP Staff member, or (c) a personal 
visit from an AFP Staff member^ Of the 423 faculty members In the total faculty 
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member sampling pool (this includes "mailed" and "personal contact" departments) 
completed questionnaires were received from 3^6 or 81. 81, 
S^s tem^w i de Samp 1 I Proc^edureg^ (73" 7^) 

Because of the attribution problem discussed in the Evaluation Plan, It 
became necessary during January of 197^ to devise a set of procedures that 
would allow a system^wide sample to be selected which could be used to collect 
FIF data Identical to that gathered from the selected departmental samples* This 
was accomplished by first excluding from the OSSHE Payroll List all faculty mem~ 
bers who were In one of the 25 previously approached and/or sampled departments. 
ThiSj In effect, left a sampling pool that Included each faculty member within 
OSSHE who was at the rank of assistant professor or above who also had a .50 PTE 
appointment In an undergraduate instructional area within the seven institutions. 
Within each Institution and for each of the three professorial ranks a serial 
number was assigned to each individual In every department eligible for inclusion 
system^wide sampling. Seven lists for each of the three professorial ranks were 
thusly developed (one for each of the seven institutions)* In this manner 614 
assistant professors, 581 associate professorsj and 640 full professors were 
identified as being eligible for Inclusion In the first mailing to the system^wide 
sample (see the fol low] ng paragraph for a description of this "first mailing" and 
the second "replacement mailing"). As with the departmental sample. Faculty 
Interview Form (FIF) packets were assembled for each selected faculty member. 
Each packet contained a cover letter requesting participation and explaining the 
purposes of the AFP Project, a sheet of directions for completing the FIF, a 
copy of the FIF, and a stamped self-addressed return envelope. 

The above-described selection process resulted In a first mailing to 
5S2 faculty members during late March of 1974. Within a period of approximately 
t\^jo and one^half weeks approximately one^third of those packets mailed in Harch 
were returned to Teaching Research. Rather than using follow-up letters (which 
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are notoriously ineffective) to raise the response rate, It was decided that 
the fnost comprehens i ve coverage of the System would be attained by mailing to 
a second group of FIF "replacement*' faculty members. These were selected by 
replacing each packet nialled In the first mailing but which had not yet been 
returned with a packet for as similar a faculty member as possible. A replace- 
ment faculty member was operationally defined by use of the 1973 05SHE Payroll 
Listing. Since all faculty members were numbered by rank within each department, 
It v^as possible to replace a non- respond i ng first mailing faculty member with the 
next numbered faculty member from the list of non-selected but eligible respon- 
dents^ For example, if assistant professor number 47 at Oregon State UnlversitfV 
had not responded after two and one-h^lf weeks, we then replaced that person 
with assistant professor number hQ from the Oregon State University list. 

This procedure had the effect of replacing first mailing non- respondents 
with a person who was quite likely to be in the same discipline at the same rank* 
In factf if the next person down on the list was not in the same departments the 
person preceding the non-respondent on the list was selected as the replacement. 
For example, If associate professor number 62 at Oregon State University had not 
responded after two and one-half weeks, we checked to see if associate professor 
number 63 was in the same department. If he v/as notj then wa would replace 
associate professor number 62 with associate professor number 61* If this 
procedure failed to obtain a replacement within the same department (i.e., if the 
department contained only one faculty member at that particular rank), then the 
replacerent became the next person down on the list regardless of department. 
A total of 386 '^replacement" packets were mailed during the second week of April. 
A final twist to this replacement procedure consisted of not using rrore than one 
completed FIF for any original ''slot" in the one-third system-^wide sample. In 
other words, if^ because a faculty member from the first group had not repsonded 
after two and one^half weeks, a replacement was sent to another individual, and 
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If after a period of six weeks both the original and the replacement for that 
one position finally were returned to Teaching Researchp then only the original 
packet was used= in the final analyses (the replacement packet was not used). 
A total of 13 "duplicate'' FIF respondents were excluded from the final analyses 
because the ''original" member of the pair was finally returned. 

Using the above-descr i bed procedures^ namely, the original mailing and the 
replacement scheme, a total of 330 completed FIF's were returned from the 
three professorial ranks. On the basis of the total number mailed (955) this 
resulted in an overall response rate of (with replacements and excluding 

duplicates)* This response^ based upon the "replacement scheme"j insured 
greater system^wlde representativeness* 

In addition to the one^third sampling procedure that has been described for 
selecting the sys tem'^^w Ide sample^ we tried to get as many FjF responses from 
department chairpersons as possible* A total of 1^6 OSSHE department chairpersons 
were identified by using both Institutional bulletins (catalogues) and the 
payroll list. We chose not to sample from among these individuals, Insteadj 
we mailed an FIF packet to each department chairperson* These packets consisted 
of essentially the same inserts as were included in the previously described 
packets for the assistant, associate, and full professors* Since we mailed to 
all available department chairpersons within the System, there was no opportunity 
for replacement. Instead of replacing as we did for the non^cha i rperons , a 
reminder letter was mailed after a period of three weeks to each non^ respond i ng 
chairperson. A total of 103 completed FIF's were returned from the group out 
of the 1^3 mailed* This resulted in an overall department chairperson response 
rate of 72, 04 , 

Professional School Sampling Procedures ilk^lS) 

Because the Year 1 data collection strategy called for a focus on faculty who 
were Involved In undergraduate instruction, coverage of faculty in professional 
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schools was not as complete as it otherwise might have been. ^ While the 

73*7^ campus analyses reported data for some professional school faculty, 

the results which compared the four discipline types indicated a need for more 

data from professional school faculty. Early in 1975 it was decided that a 

second major data collection effort would greatly enhance our understanding of 

the overall promotion/tenure process. The data collection would focus exclusively 

on professional school faculty* 

The major suppliers of data were faculty from the State System's Health 
Sciences Center (viz., Schools of Medicine, Dentistry and Nursing) and faculty 
from four professional schools at the University of Oregon (viz,, Architecture, 
Education, Journalism and Law). A slightly revised 31-1 tern FIF was used to survey 
faculty perceptions in these schools. All faculty in each setting were included 
In our data collection effort* 

Questionnaire packets were developed for each eligible faculty member. 
The packets contained a cover letter explaining the project's purposes, and FIF, 
and a stamped se I f ^addressed return envelope, A total of 363 completed forms 
were returned resulting in an overall professional school response rate of 77|, 
Follow-up Sampling Procedures (75-76) 

In an attempt to help determine the extent and nature of the project's 
impacts since its beginnings in I973ia major follow-up data collection strategy 
was developed for use during theJ4nal 8 months of the project. In the 
early spring of 1976 all faculty who repsonded during 73-74 to the original FIF 
ere targeted to receive a 34-Item third generation FIF that is hereafter referred 
to as the Faculty Perception Questionnaire (FPQ). 

Data collection began in mid-April 1976 and continued through June 10, 1976, 
A total of 485 completed FPQ's were returned from the three university and three 
college campuses of interest. Returns were also received from the Oregon 
Institute of Technology but they have been excluded from this report's analyses 



w 



88 



-77- 

because of the specialized nature of the institution* The overall follow-up 
response rate was 771, 
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Append Ix C 



Interview Questions Used with 
Presidents, Deans, and Others 
Dur I ng Fol low- up ( 1 976) 
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The following questions formed the basis for individual interviews with 
32 campus administrators during the spring of 1976. Each was asked his/her 
porception of any changes which have occurred since 1973 regarding: 

1, The evidence collected from students for the purpose of judging 
the adequacy of a promotion/tenure candidate's instruction? 

2, The relative Importance of promotion/tenure evidence pertaining 
to the adequacy of a candidate's Instruction? 

3, The clarity of the criteria used to evaluate faculty? 

The campus resources devoted to the improvement of teaching quality? 
A final question was: 

5, Are you aware of our report entitled Ca^mp u s _A n a s i s : Fa c t^rs 

Influencing Promotion/Tenure Dec Is Ions? NO YES ^ 

What influenceSs if any, did It have on the process and evidence 
used to evaluate faculty? 
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Faculty Perception Questionnaire (FPt) and 
Direct Ions for Completion 
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FACULTY PERCEPTION QUESTIONNAIRE 



TEACHING RESEARCH DIVISION 
HIGHER EDUCATION RESEARCH PROGRAM 
MONMOUTH, OREGON 97361 



FACULTY PERCEPTION QUESTIONNAIRE 
Directions for Comptiting Parts A and B: 

Part A consists of 34 factors, each of which has been used by various departments as sources of information in prfamotjon and 
tenure decision processes. No department uses all the factors, and it is very unlikely that any two would assign identical infiu- 
ence ratings to those used. The steps for compleiion are described below. 

Step 1: In Column 1 (Factor Use) check each factor thai you believe is currently used within your department when making 
promotion and/or tenure decisions. Use the space provided at the bottom of the page to add any factors that are 
used but not listed. 

Step 2; Using the 7-point scale at the top of the page, Indicate in Column 2 your perception of the present degree of influ- 
ence for each of those factors that you checked in Column 1 as being used within your department. 

Step 3: For each factor rated in Column 2 indicate in Column 3 (by circling r, 0, or I) your opinion as to whether the factor 
increased, decreased, or showed no change in influence since 1973. 

Step 4: In Column 5 consider all 34 factors plus any others you may have added, and check the 5 factors that you would 
prefer to have the greatest influence upon promotion and tenure decisions within your department. 

Step 5: Please complete the questions in Part B (on the back side of the questionnaire) and return it to us in the self- 
addressed envelope for processing. 



Example of Partially Completed Queitlonnaire— Part A 

Used But Very Very Significant 

fvtinor Influence Moderate Influerice influence 





1 a 

i 1 


3 4 

1 1 


5 


6 

I 


7 

1 




(Coi 1) 


(Col, 2) 


(Col, 3) 




(Col. 4) 




(Col. 5) 


Check {^') each faeto'' ustd m 
yeuf dsparlment 


Uiing the absvo iCile, ratt 
each eheekgd tacfor fOF 5hg 
Aiiriiani to A§SDCi3!e 
dsciSiSn 


For each riled factor^ circle 
one cheice to show ari increSis 
(J), decfiflii (]) or ns ehings 
(0) in infiuonoe sinct 1973 




FaeiQf 




Check the five facteri you 
wouid'prefsr ts be 
flissf mffuontiil 






1 0^ 


L 








T 0 i 


M 






2- 




N 






/ 




0 








T 0 i 


P 






6 


©0 i 


Q 
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Used but Very 
Minor Influence 



Faculty Perdeption Questionnaira 

Rating Scale for Part A 

Moderate Influence 
3 4 5 



Very Significant 
influence 



Cor 1 


Col 2 


Cei 3 


Col 4 




Chgck ( "/ ) 
oach fiw^or 
used in your 
depflflment 


Using the 
obovg $gilt. 

checked ftclsr 
for the 
Asiistanf 
ie Asioelata 


Fer each ratsd 
factor, Circle 
ant ehoieg is 

cfiiie (f), 
disrease (|} 
sr no chingo 
(0) in Influence 
smee !973 


PACTOR 


Cheek i^) 
the five 

factors you 
wQuld prefer 
to b@ most 

inflye ntjgi 






T 0 ; 


1, Publication In scholarly iournals and books 








0 I 


2, Student ratings of initructor performanGe 











3, Contributions to departmental committees 








0 


4. Support of departmental policy and goals 








0 


5. Assessment of course syllabi and examinations by colleagues 


_ . 






; 0 


6. Informal and impressionistic appraisal of teaching by colleagues 








T 0 


7. Formal and systematic appraisal of teaching by colleagues 


- 






t 0 I 


8. Success in attracting grant support for research and scholarship 








T 0 i 


9. Supervision of theses 








: 0 i 


10. Personality traits and general attitude 








' 0 I 


11. Consultation record on and off campus 








' 0 i 


12. Record of service on college/university/OSSHE committees 








' 0 i 


13, Academic advising 








r 0 i 


14. Membership in professional organizations 








: 0 I 


15. Service (no fee) to local and/or state community 








' 0 I 


16. Supervision of field placements (practicum students, residents or fellows) 








* 0 : 


17. Elected offices in professional organizations 








' 0 , 


18. Public and/or professional presentations of research/anistic products 








' 0 , 


19. Informal {general, impressionistic) colleague appraisal of research and/or artistic 
work 




■ - — 




' 0 I 


20. Formal (product e^camination) colleague appraisal of research and/or artistic work 








0 


21. Obtaining advanced degree 








0 I 


22. Evaluation by depariment chairman 




' 




0 


23. Informal (general, impressionistic) colleague appraisal of service work 








* 0 , 


24 Formal forfiriuct px^min^tion^ rnWmsnn^ anriraiesi f^t ssruif^i^ mr-^ey 








* 0 i 


25, Evidence of student learning in courses 










26. Election to Faculty Senate 








• 0 i 


27. Effort to remain current in discipline 








' 0 1 


28. Credit hour production 








* 0 : 


29. Student demand for course 








' 0 : 


30. Time in academic rank 








' 0 : 


31. Evaluation by school/department committee 








t 0 • 


32. Availability to students 








' 0 ; 


33. Innovative effort in teaching 






' 0 . 


^i. L^'^^l^' ^P.^ systematic appraisal of the candidate by peers outside of instriution 








' 0 , 


- - ■ - - - - ■-- - - HK — — — - 





^ Part B 

, Please circle your choica for each Item. 

1. Since 1973 what ehanges, if any, have you seen in the use of promotion tenure evidence coKected trom students regarding 
the adequacy of initructlon? 

A decrease No change from No change from Increase DK 

jn use low usage in past high usage In'past in use 

E Since 1973 what changes, if any, have you seen in the Importance of systematic evidence used to verify the adequacy of a 
promotion/tenure candidate's: 

(a) Instruction? 



A decrease In Low importance in High importance in Increased DK 

Importance past and no change past and no change importance 

(b) Research or Scholarly Development? 

A decrease In Low importance in High importance in Increased DK 

importance past and no change past and no change importance 



3, Senate Bill 413. passed in the last Oregon Legislature, completeiy revised the rules governing confidentiality of faculty 
personnel files. One of the major consequences was to prohibU anonymous judgments (except certain kinds from students) 
regarding any faculty member being considered for a promotion. 

(a) In your opinion, how has SB 413 influenced the promotfon process insofar as tha quality and thoroughness of the pro- 
motion review process? 

Improved since No visible Worse since Don't know or 

SB 413 change SB 413 not applicable 

(b) In your opinion do you think SB 413 has reduced the probability of negative votes in promotion tenure proceedings? 

Yes No DK 



EKLC 
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Table I? 

Mean FPQ Factor Values for College Faculty 
Respondents by Acfidemic Rank 



Factor Label 



Overall 

Factor 

Hein 


Overall 

Standard 

Deviation 




Acadeiiiic 1 


^ank 




Assistant 

Profs, 

(N=3B) 


Associate 
Profs. 


Full 
Profs. 


Chairmen 
(MOj 


i-i 


i.ae 


l.k 




2.7 


3.1 


io 


2.1^ 


hi 


k.l 


3.3 


^i.5 


3.2 


l.B? 


3.0 


3.2 


3,5 


3.3 


11 




L i U 


1 \ 

A.J 


") ft 


3,0 




l.]8 


.5 


• 5 


,k 


1.3 


15 


2.1? 


3.2 


3.0 


3.7 


i8 


.k 


K25 


.2 


.3 


.6 


■ ,k 


1.8 


2.2ii 


l.^t 


2.0 


1.3 


1.7 




.66 


.2 


0.0 


■ 3 


.1) 


k.\ 


2.1ii 




kA 


i2 


'ii.O 


1.3 


1.87 


1.3 


l.S 


1.0 


1.6 


Zi 
2.2 


1.35 
2.05 


IM 
2.1 


3.0 


2.7 


3.6 


2,0 


1.92 


1.3 


2.0 
2.1 


2.0 
2.0 


3.3 
2,0 


2.1 


i.ea 


2.1 


1.5 


2.2 


3.1 


1.1 


l.?8 


.8 


1.1 


.3 


2.0 


2.3 


2.01 


2.1 


2.5 


2.5 


2.0 


2,5 


2.18 


2.2 


2.8 


2.3 


3.0 


1,8 


2.12 


l.S 


1.8 


1.6 


2,5 


.2 


s 


.2 


.1 


.1 


1.0 


S.2 


2.31 




5,1 


5.2 


5.^ 


5.0 


2.i<l 


5.1 


5.0 


U 


5.1 


1.9 


2.13 


2.1 


1.7 


\.k 


2.7 


.2 


.80 


.3 


.1 


.2 


.3 


1,^ 


2,21 


i.i 


l.^i 


1.3 


2,'i 


1.0 


1.59 


.6 


1,1 


1.1 


.3 


l.k 




2.1 


2.0 


2J 




l.it 


2.1& 


1.5 


1.5 


1.^ 


1.2 ' 


1.2 


1.88 


1.0 


1.0 


1.5 


1.1 


iS 


2,30 


il 


i9 


5.0 


5.2 


3.2 


2.86 


3.2 


3.0 


3.2 


3.9 


l.k 


2.26 


2.1 


2.2 


2.6 


3.1 


2,8 


2.32 : 


2.1 


2.^ 


2,8 




,2 




.3 


0.0 


.2 


'I s 



1 Publications 

2 Student Ratings 

3 Service on Departnienteil Comniittegs 
'i Support of Department Policy and Goals 

5 Colleague Assessments ofjyllabi 

6 inforfiial Colleague Appralsals/Teiching 
1 Forfnal Colleague Appraisals/Teaching 

8 Grant Support for Research 

9 Supervision of Theses 

10 Personality Traits and Attitudes 

11 Consultation Record on/off Campus 
■12 Service on Institution/SystCT Cofuniittees 
13 Academic Advising 

1^ Hefnbership In Professional Organizations 

15 Service (no fee) to Community 

16 Supervision of Field Placements 
1? Elected Offices In Organizations 

18 Public Presentations of Products 

19 Informal Colleague Appraisals/Research 

20 Formal Colleague ADpraisals/Research 

21 Obtaining Advance Degree 

22 Evaluation by Department Chairnian 

23 Infornial Colleague Appraisals/Service 
Ik Forfnal Colleague Appraisals/Service 

25 Evidence of Student Learning In Courses 

26 Election to Faculty Senate 
11 Effort to Remain Current In Discipline 

28 Credit Hour Production 

29 Student Deniand for Course 
3D Time in Academic Rank 

31 Evaluation by School/Oepartmftnt Committee 

32 Availability to Students 
^^©''^ns^stive Effort in Teaching 

^g ER^ "»rmal Peer Appraisals (from off campus) j .§9 q!q }i \l 99 



Table 13 



Haan FP^ Factor h\um for University Faculty 
Respondents by Academic ^ank 





yvgre ! 1 


flupm 1 1 

UVs! a ! ! 




Acadeniic Rank 




Factor Label 


raCtw! 








Ful 1 C 

1 y 1 1 w 


ihii rmen 




Hisu t 3 f" j f\n 

y^V } d L lUI! 


r 1 y 1 3 i 


Profi 


Profs. { 






JJJi 




(H2) 


(fl^88) 

I 11 WW/ 






i Publications 


5.3 


2,05 


5.8 


5.5 


5.5 


3.0 


2 Student Ratings 


3.9 


2,00 


3.5 


3.9 


3.9 




3 Service on Oepartmental Coniiiilttees 


2. 6 


l.?5 


2,5 


2.5 


2,7 


2.9 • 


^1 Support of Department Policy and fioals 


1.^ 


2,02 


i,2 


1 i. 


1,6 


l.'i 


3 Colleague Assessroents or byliaDi 




1.11 




X 


.5 


,6 


S Infornial Colleague Appraisals/Teaching 


2J 


2.01 


2.6 


3.0 


2.6 


2,6 


] Forfnal Colleague Appraisals/Teaching 


1,1 


1 (-1 ^ 

1.9? 


,8 


1.1 


1.0 


1.7 


8 Grant Support for Researcn 


3-1 


2.61 


3.2 


3.^ 


3.2 


2,6 


9 Supervision of Theses 


1.8 


2.01 


1.3 




2.1 


1-9 


10 Personality Traits and Attitudes 


2.3 


2.28 


2.5 


2.6 


2.1 


2.0 


11 Consultation Record on/orf Campus 


l/l 


1.7d 


1.1 


1.0 


1.1 


1 t 

1.H 


12 Service on Instltution/Systeni Comniittees 


i.k 


1.61 


2,0 


2.i| 


2.3 


2.8 


!3 Academic Advising 


1,0 


1.9^ 


1 1. 

I,*! 


K5 


1.8 


2,8 


]k Hanibership in Professional Organizitions 


1.3 


l.?5 


l.I 


1.1 


1.5 


1,0 


15 Service (no fee) to Comniunity 


1.9 


1.88 


1.0 


1.7 


2.0 




16 Supervision of Field Placements 


.6 


1.39 


1, 

A 




.7 


.9 


IJ Elected Offices in Organizations 


1.9 


1.95 




1.8 


2.0 


2.3 


IB Public Presentations of Products 


3.3 


2.23 


3.3 


a 1 

3.1 


3.^ 


3.7 


19 Infernal Colleague Appraisals/KesBircn 


I.k 


2.32 


2.8 


2.7 


2.3 


1 A 

1.9 


20 Fomal Colleague Appraisals/Kssearch 


2.4 


2.71 


1.9 


2.4 




3.0 


21 Obtaining Advance Degree 


2.9 


3.09 


3.^ 


2.7 


2,8 


3.2 


22 Evaluation by Department Chairman 


k,k 


2,38 






kl 


kX 


23 Inrormal Colleague Appnisals/Seryice 


1.3 


1.91 


1.5 


1.2 


1.5 


.9 


Ik Formal Colleigue Appraisals/Service 


.6 


1,^(9 


.5 


.5 


.6 


.9 


25 Evidence of Student Learning in Courses 


.9 


1.81 


.8 


.9 


1.0 


1.0 


26 Election to Faculty Senate 


.6 


1.29 


X 


.7 


.7 


.6 


27 Effort to Remain Current in Discipline 


2.5 


IM 


2,0 


2.i| 


2.i| 


3.5 


28 Credit Hour Production 


.9 


1.78 


.9 


1.2 


.9 


.8 


29 Student Demand for Course 


1.3 


1,9-^ 


, 1.0 


1.3 


\X 


1.2 


30 Time in Academic Ranl< 




2.3s 


3.1 


3J 


3.7 


2.7 


31 Evaluation by School/Department Committee 


k.] 


2.78 


3.^ 


3.9 , 




\X 


32 Availability to Students 


1.5 


2.00 


1.1 


1,1 


1.6 


2.0 


^ jovative Effort itrTeachIng 


a.ii 


2.1? 


i.a 


2.1 


2,6 


2.7 " 

2.8 ' 


yMLmal Peer Appraisals (from off campus) 


2.6 


2.83 


2.5 


2.!i 


2.9 
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Table 19 

Certainty Scores for College Faculty 
by Discipline: All Ranks Combined 

The certainty scores listed below may be used to gain some sense of how consensus 
can vary by setting for each factor* Certainty scores range between 0 and 50. A 
low "certainty score*' (near zero) indicates that factor's use is clouded by uncer^ 
tainty; conversely, a high score (above 35 or 40) indicates consensus among re^ 
sponden ts regard [ ng the factor's use or non^use. 







Disclp 


1 1 nes 




Factor Label 


Arts £ 


Physl ca 1 


Social 


H rof ess ion a I 




Huran 1 t 1 es 


Sc i ences 


S c i ences 


Schoo 1 s 


1 Pubj icatlons 


29 


34 


36 


24 


L stuaent ratings 


34 


36 




H 1 


J PeiVlwc wil wcpo r Lnicrt La 1 WWlllllll LL6€b 


pu 


"a 1 
31 


32 


33 


h Support of Department Pol Icy and Goals 


o !i 
Zh 


o 


5 


24 


5 Col league Assessments of Syl labi 


1 "7 

1 3 


h3 


36 


2d 


D I n 1 o rma i uo i i o^g ue Mpp ra i 5 a i s/ i ea en i ng 






M 1 


32 


7 FornHl Colleague Appraisals/Teaching 






36 


h3 


Q urant pupport ror nesearcn 






2/ 


i, 
** 


9 Supervision of Theses 


li c 




A 1 


37 


1 u re rsona i i ty i ra i ts an a MLLicucies 


An 


27 


32 


/i ii 
***+ 


1 1 uonsuiLaLion r\e co ro on/orr uarnp us 


oL 
^4 


20 




i, 


1 ^ 3e r V 1 ce un in^Li luli on / DysLsm uornmi ttsss 




29 


23 


30 


1 * /\ #^ ^ ^ nfi i — O ^ % # i ^ i 

ij MwaUcmiu MuviSiny 


Q 

O 


1 3 


1 Q 
1 Q 




1^ Mciiiu^r3iiip III r iD 1 lOiis 1 ur ysii i luiib 


1 3 


29 


9 


1 9 


15 Service (no fee) to Communi ty 


3 




23 


22 


16 Supervision of Field Placements 


2i* 


24 


14 


2 


17 Elected Offices in Organizations 


2k 


20 


^ f 




18 Public Presentations of Products 


13 




\k 


7 


19 Informal Colleague Appraisals/Research 


8 


4 


18 


6 


20 Formal Colleague Appraisals/Research 


24 


48 


41 


48 


21 Obtaining Advance Degree 


29 


43 


23 


41 


22 Evaluation by Department Chairman 


29 


41 


27 


■ 43 


23 Informal Colleague Appraisals/Service 


13 


1 


9 


7 


2^ Formal Colleague Appraisals/Service 


29 




50 


46 


25 Evidence of Student Learning in Courses 


13 


29 


32 


2 


26 Election to Faculty Senate 


13 


13 


36 


7 


27 Effort to Remain Current i n. Di sc i p 1 i ne 


8 


20 


0 


13 


28 Credit Hour Production 


8 


34 


18 


4 


29 Student Demand for Course 


3 


29 


9 


13 


30 Time In Academic Rank 


18 


41 


32 


44 


31 Evaluation by School/Department Committee 


29 


15 


0 


4 


32 Availability to Students 


13 


13 


5 


26 


33 Innovative Effort In Teaching 


8 


27 


27 


9 


3^ Formal Peer Appraisals (from off campus) 


45 


48 


36 


46 
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Table 20 

Certainty Scores for University Faculty 
by Discipline: All Ranks Combined 

The certainty scores listed below may be used to gain some sense of how consensus 
can vary by setting for each factor. Certainty scores range between 0 and 50. A 
low ''certainty score" (near zero) indicates that factor's use is clouded by uncer-* 
tainty; conversely^ a high score (above 35 or kO) indicates consensus among re-- 
spondents regarding the factor-s use or non^use. 







Di sc I p 1 i nes 




hactor Label 


Arts £ 


Phys ical 


Social 


Profess iona 1 




Human i t i es 


Sc i ences 


Sc i ences 


Schoo 1 s 


1 Publ icafinn^ 








4b 


2 Sfudent Ratina^ 




17 
J / 


" y 




^ Spr\/inR On D^na r f rn^n f ^ 1 Cdrntfii t tf^^^ 

J \^ i V I \^ ^ \J II a 1 W 1 i rVr- 1 1 U Q ! \j\J II Mill \i l^^w^ 


k 1 




*T 1 


39 


h Sunonrf nf Di^narfmpnf Pol lev And Goa 1 s 


7 




1 1 




^ W w 1 1 ^ Si ^ / \ ^ 3 Q 33(iibll >^ 1 - / 1 1 5 W ! 










n 1 ri 1 O f 1 Pf^l l^^anii^ AnnrAi^^li /T^ a f~ H i n n . 




ill 




1 Q 


7 ForfTtfll frt 1 l^aniiR Ann rA i ^a 1 ^ / T^ar'h t n n 


^ SJ 


14 




4. M 


R nt^^nf ^linnr^rf "for r^f'h 


0 


~ \j 




o 
Z 


9 Supervis ion of Theses 


1 

1 


1 7 


1 1 
1 1 


Q 


10 Ppr^nnalifv Traif^ and Aff'ifuri^^ 






q 


U 


1 1 Con^tjl fflf ion Rpf~o rH on/off TArnnij^ 


7 1 


22 


1 7 




ly S^FV/if^^ On Inet'it'iit^iori/Sv/ef^rn Pi°iFnfnit"t"^^e 


^ 0 








1^ Academic Adx/i^ina 

1 J fi Q U fill W U V 1 ^ III ^ 


1 2 




2n 


7 


] ii Mpmhp r^hin in P ro fp^^ lonal Orcisn I ion^ 




1 ^ 


y 


1 1 
1 1 


1^ Service (no fpp) fo Commun i t v 


] ] 


7 


33 


J 1 


lA ^iin^rv/iQion ot PipIH Pl^f^^ rn^ n ^ 
lu ^ijL/wi VI3IWM v<i riwiM f is^wii 1^ i 1 L ^ 


JO 




29 


1 7 


17 Elected Offices in Organizations 


1 1 


2 


9 


26 


18 Public Pre^^nfAfion^ nf P rodur f ^ 




22 


39 


X \ 


19 Informal Colleague Appraisals/Research 


17 


19 


29 


k 


20 Formal Colleague Appraisals/Research 


8 


8 


0 




21 Obtaining Advance Degree 


5 


22 


0 


11 


22 Evaluation by Department Chairman 


36 


33 


37 


33 


23 Informal Colleague Appraisa Is/Sarvice 


5 


•II 


k 


11 


2^ Formal Colleague Appraisals/Service 


25 


37 


Ik 


32 


25 Evidence of Student Learning in Courses 


18 


31 


30 


26 


26 Election to Faculty Senate 


22 


38 


17 


33 


27 Effort to Remain Current In Discipline 


7 


15 


7 


7 


23 Credit Hour Production 


16 


33 


17 


33 


29 Student Demand for Course 


5 


21 


0 


Ik 


30 Time in Academic Rank 


29 


31 


31 


26 


31 Evaluation by School Department Committee 


21 


21 


33 


22 


32 Availability to Students 


I 


17 


6 




33 Innovative Effort in Teaching 


Ik 


6 


7 


17 


34 Formal Peer Appraisals (from off campus) 


12 


19 


3 


19 
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Table 21 

Mean FPQ Factor Values for College 
Acadefnic Disciplines - 7 point scale 







Overal 1 


Overal 1 




DIscIpl 


ines 






Factor Label 


Factor 


Standard 


Arts/ 


Physical 


Social 


Prof. 






Mean 


Devlat ion 


Humanities 


Sciences 


Sciences 


School 






(N=138) 




(N*19) 


(N*43) 


(N=22) 


(N=54) 


1 


Publ Ications 


2.6 


1.88 


3.2 


2.9 


— . ft 

2,8 


2.2 


z 


Student Ratings 


k.O 


2.15 


4.3 


3.8 


k.B 


3.8 


3 


Service on Departmental Commfttees 


3.2 


1.88 


3.9 


3.1 


2,8 


3.3 


k 


Support of Department Policy and Goals 


2.7 


2.33 


2.9 


2.0 


2,0 


3.4 
,8 


5 


Colleague Assessmenta of Syllabi 


.6 


1.40 


1.2 


.1 




6 


Informal Colleague Appraisals/Teaching 


3.5 


2.17 


3.2 


3.4 


1 


3.6 


7 


Formal Colleague Apprfilsals/Teaching 


.k 


1.24 


.7 


.2 


.5 


.3 


8 


Grant Support for Research 


1.8 


2.26 


.9 


2.6 


hO 


2.0 


9 


Supervision of Theses 


.2 


.68 


.1 


.1 


.1 


.4 


10 


Personality Traits and Attitudes 


k. 1 


2.14 


3.8 


3.2 


5.0 


1 1 


Consultation Record on/off Campus 


1.3 


1.84 


.6 


.8 


IJ 


1.9 


12 


Service on Institution/System Committees 


2.8 


1.92 


3.0 


2.9 




2.9 


13 


Academic Advis ing 


2.2 


2.06 


1.2 


1.7 


2.6 


2.9 


H 


Membership In Professinnal Organizations 


2.0 


1.93 


2.1 


2.3 


.9 


2.3 


15 


Service (no fee) to Community 


2. 1 


1 .89 


1.7 


1.9 


2.2 


2,4 


16 


Supervision of Field Placements 


1.1 


1.79 


.6 


,6 


1.2 


1.7 


17 


Elected Offices In Organizations 


2.3 


2.02 


2.6 


2,0 


1.3 


1, 1 


18 


Public Presentations of Products 


2.5 


2.18 


2.8 


3.1 


2.0 


2.2 


19 


Informal Colleague Appraisals/Research 


1.8 


2.15 


2.4 


1.6 


1.0 


2.1 


20 


Formal Colleague Appraisals/Research 


.2 


.99 


.9 


.1 


.3 


.2 


21 


Obtaining Advance Degree 


5.2 


2.32 


4.6 


5.4 


4.4 


5.7 


22 


Evaluation by Department Chairman 


5.0 


2.42 


4.6 


5.3 


3.7 


5.4 


23 


Informal Colleague Apprs 1 sal s/Servi ce 


1.9 


2.13 


1.2 


1.7 


1.6 


2.3 


2k 


Formal Colleague Appraisals/Service 


.2 


.81 


.6 


.1 


0.0 


.1 


25 


Evidence of Student Learning in Courses 


1.^* 


2.22 


1.6 


.9 


.7 


2.1 


26 


Election to Faculty Senate 


1.0 


1.61 


.9 


1.1 


.4, 


1.2 


27 


Effort to Remain Current in Discipline 


2.k 


2.45 


1.8 


2.7 


2.0 


2.8 


28 


Credit Hour Production 


1.4 


2.19 


1.3 


.3 


1.4 


2.3 


29 


Student Demand for Course 


1.2 


1.90 


1.7 


.6 


1.7 


1.3 


30 


Time in Academic Rank 


k.B 


2.28 


3.3 


4.8 


4.9 


5.3 


31 


Evaluation by School /Department Committee 


3.2 


2.85 


4.4 


3.3 


2,8 


3.0 


32 


Aval labi 1 ity to Students 


2.4 


2.26 


1.1 


2.4 


2.4 


3.1 


33 


Innovative Effort in Teaching 


2.8 


2.33 


2.2 


3.Z 


3.2 


2.6 
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Formal Peer Appraisals (from off campus) 


.2 


1.02 


.2 


.1 


.7 





I 

/ 



ERIC 



Tibli 22 



Hean FFQ Factor Values for University 
Acidemic Disciplinis ■ ] point scale 



Overall Overall 



PisciE 



ines 



Factor Label 



r Publications 

2 Student Ritings 

3 Service on Depirtmental Conimittees 

'i Support of Department Policy and Goals 

5 Colleague Aisessnients of Syllabi 

6 Infornial Colleigue Appralsals/Teiching 
] Formal Colleague Appriisi I s/Teicliing 

8 Grant Support for Researcli 

5 Supervision of Theses 

10 Personality Traits and Attitudes 

11 Consultation Record on/off Cimpus 

12 Service on Institution/System Committees 

13 Academic Advising 

\k Hembership in Professional Organizations 

15 Service (no fee) to Comitiiinity 

16 Supervision of Field Placements 
1?. Elected Offices in Organizations 
18 Public Presentations of Products 

13 Informal Colleague Appraisals/Research 

20 Fornial Colleague Appraisal s/Researcii 

21 Obtaining Advance Degree 

22 Evaluation by Departnient Chairman 

23 Informal Colleague Appraisals/Service 
Ik Formal Colleague Appraisals/Service 

25 Evidence of Student Learning in Courses 

26 Election to Faculty Sonata 

21 Effort to Reoiain Current in Discipline 

28 Credit Hour Production 

25 Student Defnand for Course 

30 Time In Academic hnk 

31 Evaluation by School /Departinent Committee 

32 Availability to Students 



r 'nnovatlve Effort in Teaching 
'ERjCoriiial Peer Apppaisals (from off 



caiiipus 



Factor 
Hein 


Stindard 
Deviation 


Arts/ , 
Human i tie 

m) 


- Physicil 
% Sciences 


Social 
Scienc 
(N-70) 


Prof, 
es School 
imk) 


5.5 


2.03 


5.2 


6.3 


5.? 


k6 


3.8 


2.01 


i5 


3.1 
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